©2026 adebench · adecubedbuilt 2026-11-04 from the reports in the repo

← leaderboard

memU

personal memory kept as wiki pages, written by an external agent on memU's own jobs. https://github.com/NevaMind-AI/memU

Core
59% · 32.2 / 55 (mean)
Core (public2)
public2 run in progress
Private set
after the public2 runs
Full
65% · 42.1 / 65 · 65 of 100 measured
Runs
5 · core 32.2–32.2 · the details below are the median run
Common with gbrain
42.2 vs 50.4 on 65
Chars per answer
3,543 · answer after 742
Write → visible
532 ms · p95 532 ms
Write-back
0 of 2 degraded answers came back
Reached through
CLI / Python library
Model on write
yes: the executor agent (Gemini)
Key needed
LLM provider
Where it runs
local
License
Apache-2.0
Deterministic
yes in five builds
Configuration
Versionmemu-cli 0.11.0b3 from source (git main)
Modeself-hosted library, SQLite store, no memu.so account; memU runs no model, an external executor agent writes; run through a JSON-lines bridge
Doorprogressive_retrieve(question), printed as `memu retrieve` prints it (JSON, indent 2): segments, their files with full content, resources
Resultstop-5 segments (memU's own retrieve config, not changed), no relevance floor, no cut
Model insidegemini-3-flash-preview (Google, OpenAI-compatible endpoint, function calling) as memU's external executor agent: writes and patches the wiki pages; no model on read
Embeddingsgemini-embedding-001, 3072 dimensions (Google, through memU's openai provider with Google's base URL)
StorageSQLite plus a markdown workspace of recall files
Not defaultexecutor agent is Gemini with memU's own prompt and five workspace-only file tools instead of a shell (memU leaves the executor to the caller); embeddings through MEMU_EMBED_PROVIDER=openai with MEMU_BASE_URL set to Google; memU's own MEMU_EMBED_MODEL cleared by the bridge; workspace under the store instead of the CLI's fixed one in the home; telemetry, template and docs refresh off (reason not stated); built from source because the PyPI wheels do not ship memorize

Notes: 0.11.0b3 from source; memU runs no model itself, so Gemini 3 Flash plays the executor agent with memU's prompt and workspace-only file tools (the score is memU plus this executor); gemini-embedding-001. Run 2026-09-30, door retrieve, fingerprint 1e4728926b. Report: examples/memu_report/reference.md · All runs: examples/memu_report/repeats.json

Sections
door
18.8
of 25 · 6 pass · 2 fail
FAIL Which version of MailBridge is installed for the mailbox? — STALE value delivered next to the current one: 1.3.0
FAIL What is the owner's phone number? — missing +39
cards
not measuredsection not measured: no entity cards in this memory
updates
6.7
of 10 · 2 pass · 1 fail
FAIL restating the current value does not pile up a second copy — 2 copies delivered
time
0.0
of 10 · 0 pass · 2 fail
FAIL semantic memories carry their age (0/40) — 0%
FAIL an imported memory reaches the door with its original date — imported memory not served within 30 s
live state
10.0
of 10 · 7 pass · 0 fail
abstention
6.7
of 10 · 0 pass · 4 fail
FAIL What is the Zarpetta module? — 5 keyword facts for something that does not exist
FAIL Who is Ottavio Brambillesco? — 5 keyword facts for something that does not exist
FAIL Which port does the Fulmicotone service use? — 5 keyword facts for something that does not exist
FAIL What does the calendar plugin Girandola do? — 5 keyword facts for something that does not exist
file search
not measuredsection not measured: no repo configured (--repo)
graph
not measuredsection not measured: no case with evidence (all SKIP or no cases)