The testing layer for agent memory
Your agent remembers.
Can you trust what
it remembers?
Catch cross-user memory leaks, deletion failures and memory regressions with reproducible tests built for AI agents. Every finding comes with the evidence to reproduce it.
pip install memtru · runs locally · no account needed
- FAILCross-user isolationMT-ISO-001
- PASSCross-tenant isolationMT-TEN-002
- PASSDeletion visibilityMT-DEL-003
- 02searchtenant-a/user-a → 1 result(s); fixture present
- 06searchtenant-a/user-b → 1 result(s); fixture present
- 07read_by_idtenant-a/user-b → returned fixture
Persistent memory adds failure modes your other tests miss.
Unit tests check functions. Observability shows what happened. Neither tells you whether memory written for one user can surface for another, or whether a delete actually removed anything from retrieval.
- 01One user's memory is returned to another user.Typical cause: A search path filters by tenant but forgets the user.
- 02Deleted memory is still retrievable.Typical cause: The delete sets a flag that the search index never reads.
- 03An old preference outranks the new one.Typical cause: Updates are inserts, so both facts stay active.
- 04Session context leaks into other sessions.Typical cause: Session scope is stored but never applied on read.
- 05An update rewrites someone else's record.Typical cause: The write matches a shared key without the owner filter.
Three steps, run wherever your tests run.
- Step 1
Connect a test adapter
Point Memtru at a sandbox: the built-in store, any service that implements the REST contract, or your own Python adapter class.
- Step 2
Run controlled scenarios
Each scenario writes synthetic fixtures, proves the authorized path can see them, then probes the boundary. No proof of visibility, no verdict.
- Step 3
Review and prevent regressions
Read the finding, reproduce it, fix it, re-run. Compare runs in CI so a resolved failure cannot quietly return.
Six storage scenarios available today.
Each has a documented contract, a positive control, bounded retries and scoped cleanup. Severity follows a fixed mapping, so a failure is never labelled critical just because it failed.
| Scenario | What it checks | If it fails |
|---|---|---|
User isolation MT-ISO-001 | A second user in the same tenant cannot retrieve the first user's memory. | high |
Tenant isolation MT-TEN-002 | No principal in tenant B, including one with the same user id, sees tenant A's memory. | critical |
Deletion visibility MT-DEL-003 | A deleted record stops being returned within a bounded consistency window. | high |
Stale memory MT-STALE-004 | After an update, the new value is the current fact; history is marked as history. | medium |
Session boundaries MT-SES-005 | Session-scoped memory is not visible from another session. | medium |
Memory integrity MT-INT-006 | Updating one record leaves unrelated records, metadata and ownership unchanged. | high |
Also included: MT-POISON-007 (agent-mode memory poisoning, requires an agent adapter you provide), MT-REG-008 (comparison between two runs), and Memtru Afterlife (MT-AF-001, MT-AF-002: deleted memory that returns after a late write or a new session; requires lifecycle hooks in your adapter).
A CLI first. Exit codes your CI understands.
init, doctor, test. JSON for machines, a self-contained HTML report for people. Exit code 1 means an assertion failed, 3 means the run could not be completed, never the other way round.
$ memtru test
Memtru support-agent · adapter in_memory · sandbox · 7 scenarios
✕ FAIL MT-ISO-001 Cross-user isolation 1 ms
Another user retrieved a memory outside its authorized scope.
✓ PASS MT-TEN-002 Cross-tenant isolation 1 ms
✓ PASS MT-DEL-003 Deletion visibility 0 ms
✓ PASS MT-STALE-004 Stale memory 1 ms
✓ PASS MT-SES-005 Session isolation 1 ms
✓ PASS MT-INT-006 Memory integrity 2 ms
– SKIPPED MT-POISON-007 Controlled memory poisoning 0 ms
Skipped: the adapter does not provide AGENT_RESPOND.
Result ✕ FAIL 5 passed, 1 failed, 0 inconclusive, 0 errors, 1 skipped pass rate 5/6 executed
Run ID 9c6a45d5-3c97-4480-97be-942915ea1d77
Findings 1
[high] Cross-user memory returned outside authorized scope (MTF-9c6a45d5-ISO-001)
JSON .memtru/reports/9c6a45d5-3c97-4480-97be-942915ea1d77.json
HTML .memtru/reports/9c6a45d5-3c97-4480-97be-942915ea1d77.htmlKnow what passed, failed, or could not be verified.
The same comparison the dashboard shows, rendered from the two captured demo runs: before and after the user filter was restored.
- Resolved 1
- Unchanged 6
| Scenario | Before | After | Change |
|---|---|---|---|
Cross-user isolation MT-ISO-001 | FAIL | PASS | Resolved |
Deletion visibility MT-DEL-003 | PASS | PASS | Unchanged |
Memory integrity MT-INT-006 | PASS | PASS | Unchanged |
Controlled memory poisoning MT-POISON-007 | SKIPPED | SKIPPED | Unchanged |
Session isolation MT-SES-005 | PASS | PASS | Unchanged |
Stale memory MT-STALE-004 | PASS | PASS | Unchanged |
Cross-tenant isolation MT-TEN-002 | PASS | PASS | Unchanged |
Configuration differences
These runs were not configured identically. A changed result can come from configuration rather than code.
- adapter.faults: ignore_user_scope → none
- configuration fingerprint changed
Your test data stays local by default.
The CLI runs on your machine or CI runner and has no telemetry. Uploading a report is a separate, explicit command that shows you what will be sent first.
Fingerprints, not content
Reports describe fixtures by hash. Raw content is opt-in and stripped before any upload.
Redaction, twice
Common credential formats are redacted by the CLI and rejected again by the server.
Scoped cleanup
Only records created by the run are deleted. A failed run never triggers a broad wipe.
Explicit targets
Remote targets need confirmed authorization; metadata and link-local addresses are refused.
What connects today, and what does not yet.
Available
- in_memoryBuilt-in reference store with fault injection, for learning and demos.
- httpAny service implementing the documented REST contract. Verified against a reference server.
- mem0OSS SDK 2.2.1, verified with local Qdrant. Tests search filters; hosted API verification is separate.
- customYour own Python class implementing the adapter contract.
- GitHub ActionsCopyable workflow; reports kept as build artifacts.
Planned, not available
- Provider-specific adapters for hosted memory services
- Built-in agent adapters for agent-mode scenarios
- Pull request comments and JUnit/SARIF output
Test memory boundaries before they fail in production.
The local quickstart takes a few minutes and needs no account; create one when you want a shared dashboard.