Test your first real AI agent with Memtru

Go from a separate application to a reproducible deleted-memory failure, a fix and a passing report. The downloadable starting point uses two deterministic agent handlers, durable SQLite jobs and a separate worker. No Memtru account, repository access, model keys or cloud resources are required.

First distinguish the two steps: running this example verifies an actual HTTP and process boundary with synthetic data. Testing your agent requires mapping the same operations to your authorized sandbox. Downloading it does not scan your application automatically. The example does not run an LLM.

1. Install and download

Requires Python 3.11+. Download the standalone example and extract it. Its SHA-256 manifest lists the seven included files. Review the source before running it.

From the extracted memtru-first-agent directory:

python -m venv .venv

Activate it on Windows PowerShell:

.\.venv\Scripts\Activate.ps1

Or on macOS/Linux:

source .venv/bin/activate

Then:

python -m pip install memtru==0.1.1
python prove.py --repeat 1 --output-dir ./reports

Open reports/index.html locally. If that directory exists, choose a new output path; the runner refuses to overwrite previous evidence. Do not serve the source directory or credentials over a public web server.

2. Read the first failure and fix

File	What it shows
reports/broken/latest.html	MT-AF-001 FAIL: the worker committed its captured copy after deletion
reports/fixed/latest.html	MT-AF-001 PASS: the same worker processed the job but checked the deletion tombstone inside its write transaction
reports/verification.json	Expected outcomes, cleanup, negative controls and evidence references
reports/fixed-application-audit.json	Synthetic lifecycle events and actual process identifiers

The runner checks both variants and calls the existing CLI. CLI failure exits 1; the corrected test exits 0. The proof runner itself exits zero only when all its expected outcomes, including the deliberately failing target, were observed. The original report comparison must resolve the same finding.

The minimal fix is in customer_service.py: the fixed worker checks the tombstone and writes in one transaction. Removing a record without invalidating an already captured job is insufficient. Your system may need a version/epoch check or revocation event instead; do not copy SQLite locking assumptions into a distributed system without checking its own concurrency contract.

3. Map your sandbox, not a second simulator

Keep one engineering owner, an explicit synthetic namespace and a cleanup owner. Start with the ordinary deletion check if you cannot control an asynchronous job. Do not claim an Afterlife result until the real worker can acknowledge the event.

Adapter operation	Must reach your application
add, search, get, delete	The actual memory write/retrieval/deletion paths, with your real authorization mapping
begin_pending_write	An actual update captured before deletion and paused before commit
release_pending_writes	Resume that exact job and wait for the worker's acknowledgement
inspect_surface	Read-only observations of the agreed downstream stores; querying must not recreate data
teardown	Exact synthetic ids/jobs in the run namespace; never a provider-wide reset

Read http_adapter.py for the contract mapping and customer.yaml for the existing scenario. Its loopback-only client is intentionally restricted; it cannot be repurposed by merely changing a hostname. For a remote sandbox, use the supported HTTP adapter and lifecycle endpoints, with HTTPS, explicit authorization, an allowlisted host and scoped credentials in the environment. Alternatively implement a reviewed custom adapter in your own repository.

Your service must derive user/tenant identity from the real authentication path. Trusting caller-chosen test headers verifies storage filters only. Declare only capabilities you implement: unsupported prerequisites must SKIP, missing positive controls must be INCONCLUSIVE, and transport failures must be ERROR.

4. Run your config and compare

Place your reviewed adapter/config together in your sandbox repository. Set only the environment variables required by your adapter using your secret store, then:

memtru doctor --config customer.yaml
memtru test --config customer.yaml --output-dir ./before

After your application team fixes the observed behavior:

memtru test --config customer.yaml --output-dir ./after
memtru compare ./before/latest.json ./after/latest.json

Keep scenario, adapter, principals, timing and surfaces stable for comparison. Review the reports locally before opting into any dashboard upload. See the CLI reference for authentication and upload commands.

5. Record whether it was useful

Record elapsed integration time, supported/skipped paths, time to first valid report, actionable findings, reproduction success, cleanup and retest outcome. Repeat in your CI after changes to deletion, caches, authorization or workers. Use the evaluation worksheet to separate actual customer results from synthetic evidence. If required access is missing, record the blocker instead of inventing a PASS.

Need help mapping one workflow? Send an evaluation request. No charge or engagement is created by running this example or submitting the form. A PASS holds only for the configured paths and timing; it does not prove physical erasure, model safety, compliance or absence of every memory leak.