Model
gemini-2.5-flash
4 of 5 tasks passed, in 10,305 ms of wall time and 1,104 thinking tokens.
- Tasks passed
- 4 of 5
- Total time
- 10,305 ms
- Thinking tokens
- 1,104
- One-word probe
- 1,139 ms
- Not asked
- 4 (quota)
| Task | Result | Time | Thinking tokens | What came back | |
|---|---|---|---|---|---|
| Output the number and nothing else | pass | 1,315ms | 118 | clean | |
| Return valid JSON that matches a schema | pass | 2,107ms | 252 | population=9700000 | |
| Find one fact buried in a long document | pass | 2,225ms | 81 | found | |
| Hold three format rules at once | pass | 2,262ms | 326 | 3×4, capitalised | |
| Answer in exactly seven words | fail | 2,396ms | 327 | 6 words |
Findings
Where it failed
Answer in exactly seven words — 6 words
It reports thinking tokens
1,104 across 5 tasks. Those are billed, and they land before the first output token — which is why wall time and output length disagree on this model.
4 tasks could not be asked
The free-tier quota was spent before we got to Write a sentence without the letter E, Return an array of exactly five items, Reverse a list exactly, Four-step arithmetic, number only. Those are left out of the pass rate rather than counted against the model.
How this was measured
Every task was sent at temperature 0, one request at a time, and judged by code — the checkers live in bench/tasks.mjs. Failed responses are stored verbatim so the verdict can be re-read. See all models and tasks.