Did it do the right thing?
We inspect saved events, memory, source reads and write attempts. A confident answer does not count as proof of action.
The same everyday tasks. Different models.
Compare what they say, what they do, and what it takes.
Published runs of our Danish personal assistant benchmark.
Loading published results…
Each row uses this published run only. Failed tasks remain in the pass rate. Latency is for passing tasks.
| Model | Tasks passed | Median time | Reviewer score | Inference cost |
|---|
We inspect saved events, memory, source reads and write attempts. A confident answer does not count as proof of action.
A separate model reviews the answers and actions for correctness, authority, Danish clarity and unnecessary effort. Its assessment is an opinion, shown beside the hard checks.
We record task duration, model calls and reported inference cost. Unknown prices remain unknown. Infrastructure and external-service costs are excluded.
We use the real agent loop with synthetic mail, calendar and browser services. These timings exclude mobile delivery, queueing and actual browser startup. They do not measure production performance.
This site serves published snapshots only. It has no benchmark runner, inference keys, upload endpoint or execution API. We run and review benchmarks separately, then deliberately publish selected results.