Model
gemini-3.1-flash-lite
8 of 9 tasks passed, in 9,899 ms of wall time, with no thinking tokens reported.
- Tasks passed
- 8 of 9
- Total time
- 9,899 ms
- Thinking tokens
- none reported
- One-word probe
- 577 ms
| Task | Result | Time | Thinking tokens | What came back | |
|---|---|---|---|---|---|
| Write a sentence without the letter E | pass | 737ms | not reported | 8 words, clean | |
| Return an array of exactly five items | pass | 762ms | not reported | red,blue,green,yellow,purple | |
| Hold three format rules at once | pass | 782ms | not reported | 3×4, capitalised | |
| Return valid JSON that matches a schema | pass | 923ms | not reported | population=9776000 | |
| Answer in exactly seven words | pass | 1,048ms | not reported | 7 words | |
| Find one fact buried in a long document | pass | 1,116ms | not reported | found | |
| Four-step arithmetic, number only | fail | 1,314ms | not reported | got "188" | |
| Reverse a list exactly | pass | 1,488ms | not reported | exact | |
| Output the number and nothing else | pass | 1,729ms | not reported | clean |
Findings
Where it failed
Four-step arithmetic, number only — got "188"
It reports no thinking tokens
Every response came back without a thoughtsTokenCount field. That is not the same as a measured zero: the API simply does not report one here, so cost models that read that field see nothing.
How this was measured
Every task was sent at temperature 0, one request at a time, and judged by code — the checkers live in bench/tasks.mjs. Failed responses are stored verbatim so the verdict can be re-read. See all models and tasks.