Task
Write a sentence without the letter E
A constraint the model must hold across every token it writes. The most common letter in English.
- Measured
- 2026-09-12 UTC
- Result
- 2 of 3 passed · 3 not asked
- Temperature
- 0
- Prompt size
- 112 chars
| Model | Result | Time | Thinking tokens | What came back | |
|---|---|---|---|---|---|
| gemini-2.5-flash | not asked | 213ms | not reported | 429 RESOURCE_EXHAUSTED | |
| gemini-2.5-flash-lite | not asked | 218ms | not reported | 429 RESOURCE_EXHAUSTED | |
| gemini-3.5-flash | not asked | 377ms | not reported | 429 RESOURCE_EXHAUSTED | |
| gemini-3.1-flash-lite | pass | 737ms | not reported | 8 words, clean | |
| gemini-3.5-flash-lite | fail | 945ms | not reported | 1 × "e" | |
| gemini-3-flash-preview | pass | 13,869ms | 3,930 | 6 words, clean |
What happened
3 models could not be asked
gemini-2.5-flash returned 429 RESOURCE_EXHAUSTED
gemini-2.5-flash-lite returned 429 RESOURCE_EXHAUSTED
gemini-3.5-flash returned 429 RESOURCE_EXHAUSTED.
That is a spent quota, not a wrong answer — it is left out of the pass rate rather than counted as a failure.
1 of 3 models failed
gemini-3.5-flash-lite — 1 × "e"
65.1× between fastest and slowest
gemini-2.5-flash finished in 213ms. gemini-3-flash-preview took 13,869ms for the same prompt.
The exact prompt
Describe the ocean in one sentence. Do not use the letter 'e' anywhere in your answer. Output only the sentence.
Pass/fail is decided by code, not by reading the answer. The check for this task is in bench/tasks.mjs, and failed responses are stored verbatim so the verdict can be re-read.