Model
gemini-3.5-flash-lite
7 of 9 tasks passed, in 8,170 ms of wall time, with no thinking tokens reported.
- Tasks passed
- 7 of 9
- Total time
- 8,170 ms
- Thinking tokens
- none reported
- One-word probe
- 764 ms
| Task | Result | Time | Thinking tokens | What came back | |
|---|---|---|---|---|---|
| Four-step arithmetic, number only | fail | 682ms | not reported | got "120" | |
| Output the number and nothing else | pass | 765ms | not reported | clean | |
| Reverse a list exactly | pass | 797ms | not reported | exact | |
| Answer in exactly seven words | pass | 906ms | not reported | 7 words | |
| Write a sentence without the letter E | fail | 945ms | not reported | 1 × "e" | |
| Hold three format rules at once | pass | 945ms | not reported | 3×4, capitalised | |
| Return valid JSON that matches a schema | pass | 1,019ms | not reported | population=9776000 | |
| Return an array of exactly five items | pass | 1,047ms | not reported | Red,Blue,Green,Yellow,Purple | |
| Find one fact buried in a long document | pass | 1,064ms | not reported | found |
Findings
Where it failed
Write a sentence without the letter E — 1 × "e"
Four-step arithmetic, number only — got "120"
It reports no thinking tokens
Every response came back without a thoughtsTokenCount field. That is not the same as a measured zero: the API simply does not report one here, so cost models that read that field see nothing.
How this was measured
Every task was sent at temperature 0, one request at a time, and judged by code — the checkers live in bench/tasks.mjs. Failed responses are stored verbatim so the verdict can be re-read. See all models and tasks.