ADEBENCH — agent memory benchmark and leaderboard
A benchmark for agent memory. It scores the text a memory actually delivers to the model, on one golden set, with no LLM judge.
| # | Memory | Core · 55 | Full |
|---|---|---|---|
| 1 | Dakera |
94% 51.9 / 55 · 1 run |
97% 96.9 / 100 |
| 2 | ADE Brain the memory of adebench's author, and its code is private: nobody else can rerun this row. It runs on the same set and rules as every other memory |
89% 48.8 / 55 · same in 5 runs |
94% 93.8 / 100 |
| 3= | Aionforge |
76% 41.8 / 55 · 1 run |
80% 51.8 / 65 |
| 3= | supermemory |
74% 40.5 / 55 · 38.1–43.0 · 5 runs |
78% 50.5 / 65 · 48.1–52.9 |
| 3= | gbrain |
73% 40.4 / 55 · same in 3 runs |
84% 75.4 / 90 |
| 5 | Hindsight |
66% 36.3 / 55 · 1 run |
66% 36.2 / 55 |
| 6= | agentmemory |
65% 35.5 / 55 · same in 3 runs |
66% 42.6 / 65 |
| 6= | agentmemory (Gemini) |
65% 35.5 / 55 · same in 5 runs |
66% 42.6 / 65 |
| 6= | engram |
64% 35.2 / 55 · same in 3 runs |
70% 45.2 / 65 |
| 6= | Memoose |
63% 34.6 / 55 · same in 3 runs |
76% 68.3 / 90 |
| 7= | cognee |
63% 34.4 / 55 · 33.7–35.4 · 5 runs |
68% 44.4 / 65 · 43.7–45.4 |
| 7= | mem0 |
62% 34.2 / 55 · 30.4–35.5 · 5 runs |
68% 44.2 / 65 · 40.4–45.4 |
| 12= | memU |
59% 32.2 / 55 · same in 5 runs |
65% 42.1 / 65 |
| 12= | Nemp |
58% 32.1 / 55 · 1 run |
56% 36.4 / 65 |
| 12= | Jev-Mem |
58% 31.9 / 55 · 28.9–33.0 · 6 runs |
64% 41.9 / 65 · 39.0–42.9 |
| 12= | TokenMizer |
57% 25.6 / 45 · same in 5 runs · no time |
57% 25.6 / 45 |
Core is what every memory can be measured on, a write and a read through the door: door 25, updates 10, time 10, abstention 10. The ranking is on it. Full adds what a memory has behind the door (cards, live state, file search, graph), over the points it could be measured on. Chars per answer is what the model receives for one question: the same score at 300 characters and at 20,000 is not the same memory. Write → visible is how long a value just written takes to reach the door. Scores are the mean of the runs kept for each memory, with their range; two memories closer than the wider of their ranges (never less than one probe, 1.8 points) share a rank, marked =, and no memory ranks above one with a higher mean.
| Memory | Score | Probes |
|---|---|---|
ADE Brain |
98% 98.0 / 100 |
165 set 9a276b567d |
The golden set is small on purpose, so it runs anywhere. The bench was built for something else: a memory measured on its owner's data, with probes written on the facts it really holds, rerun after every change. Not comparable across memories, and the number that tells an owner whether a change helped. Totals only: the probes are the owner's facts.
Method
Eight sections: door, cards, updates, time, live state, abstention, file search, graph. A point is earned when the expected words are in the text the model receives; a retired value next to the current one is a failure.
Run it
No dependencies. Every adapter and importer is in the repository, and each memory's page names the configuration that produced its numbers.
python -m adebench --help
Your memory
Write your probes, point the harness at your memory, send the totals with a pull request. The set's hash ties a number to the probes that produced it.
GitHub
adebench is MIT. Adapters, golden set, reports: adecubed/adebench. Each memory's author is told when its results go up; if we ran it wrong, we fix it and rerun.