mem0
memory layer for agents: an LLM extracts facts from each exchange, hybrid vector and BM25 search. https://github.com/mem0ai/mem0
- Core
- 62% · 34.2 / 55 (mean)
- Core (public2)
- public2 run in progress
- Private set
- after the public2 runs
- Full
- 68% · 44.2 / 65 · 65 of 100 measured
- Runs
- 5 · core 30.4–35.5 · the details below are the median run
- Common with gbrain
- 44.8 vs 50.4 on 65
- Chars per answer
- 2,311 · answer after 97
- Write → visible
- 511 ms · p95 557 ms
- Write-back
- 0 of 2 degraded answers came back
- Reached through
- Python library
- Model on write
- yes: an LLM extracts every write
- Key needed
- LLM provider
- Where it runs
- local
- License
- Apache-2.0
- Deterministic
- no: extraction is a model call
Configuration
| Version | mem0ai 2.2.1 |
| Mode | open-source library, not the hosted platform, run through a JSON-lines bridge in mem0's own Python |
| Door | Memory.search(query, filters={user_id: "adebench"}, top_k=20), one line per result with its metadata |
| Results | top_k 20 (mem0's default), threshold 0.1 (mem0's), no cut |
| Model inside | gemini-3-flash-preview (Google, mem0's gemini provider): extracts memories from every add (infer=True, ADD-only) |
| Embeddings | models/gemini-embedding-001, 768 dimensions (Google) |
| Storage | local Qdrant on disk (vectors, BM25 slot, entity collection), SQLite history |
| Not default | max_tokens 8192 instead of 2000 (the default counts Gemini 3's thinking tokens and cut extractions, items silently lost); Gemini provider for LLM and embedder instead of the OpenAI default, other cloud keys removed so nothing falls back to it; telemetry off (MEM0_TELEMETRY=False, reason not stated) |
Notes: open-source library 2.2.1 (not the hosted platform), gemini-3-flash-preview (max_tokens 8192: the 2000 default cut Gemini's thinking and dropped extractions), gemini-embedding-001, local Qdrant. Run 2026-09-30, door search,
fingerprint 0806cb66a8. Report:
examples/mem0_report/reference.md · All runs: examples/mem0_report/repeats.json
Sections
door | 18.8 of 25 · 6 pass · 2 fail | FAIL Which version of MailBridge is installed for the mailbox? — STALE value delivered next to the current one: 1.3.0 FAIL What is the owner's phone number? — missing +39 |
cards | not measured | section not measured: no entity cards in this memory |
updates | 6.7 of 10 · 2 pass · 1 fail | FAIL a new write with a changed value replaces the old one at the door (no id given) — new value served in 621 ms; STALE: the old value is still delivered (20 reads with both) |
time | 6.0 of 10 · 3 pass · 2 fail | FAIL semantic memories carry their age (0/88) — 0% FAIL an imported memory reaches the door with its original date — imported memory not served within 30 s |
live state | 10.0 of 10 · 7 pass · 0 fail | |
abstention | 3.3 of 10 · 0 pass · 4 fail | FAIL What is the Zarpetta module? — episodes presented as direct matches; 11 keyword facts for something that does not exist FAIL Who is Ottavio Brambillesco? — episodes presented as direct matches; 11 keyword facts for something that does not exist FAIL Which port does the Fulmicotone service use? — episodes presented as direct matches; 11 keyword facts for something that does not exist FAIL What does the calendar plugin Girandola do? — episodes presented as direct matches; 11 keyword facts for something that does not exist |
file search | not measured | section not measured: no repo configured (--repo) |
graph | not measured | section not measured: no case with evidence (all SKIP or no cases) |