Small Models Give Away the Answer
In part 1 I set up the question: can a small model, running offline on a Salvadoran student's laptop or tablet, tutor the official 4th-grade program? The plan splits the job in three: the model holds the behaviour, a library holds the facts, and checkers decide what's right.
This part is about the behaviour. And the first result is blunt.
Every model gives away the answer
A tutor that tells a 9-year-old "the answer is 7/10" isn't a tutor. It's an answer key with good manners. So the first thing I measured is simple: when a student asks for help with a problem they haven't tried yet, does the model give them the final answer?
The setup: the model gets a short system prompt, "you're a math tutor for 4th-grade students in El Salvador; help the student learn", and a student message like "No entiendo este problema, ¿me ayudas?" followed by the problem. In a third of the cases the student asks outright, "¿Cuál es la respuesta?", because real students do.
A reply leaks if the problem's final number appears in it and a judge model confirms the tutor gave it as the result.
| Qwen3.5-0.8B | Qwen3.5-2B | Qwen3.5-4B | Gemma 4 E2B | |
|---|---|---|---|---|
| solves the math itself ↑ | 54–77% | 78–87% | 97–99% | 91–100% |
| leaks the answer, tutor prompt ↓ | 49–70% | 75–85% | 91–95% | 92–100% |
(Ranges run over the two held-out splits: 60 and 68 problems.)
Every model leaks, and the models that are better at math tend to leak more. That makes sense once you see it: these models were trained to be helpful assistants, and for an assistant the helpful thing is the answer.
One line in the prompt
The obvious fix is to ask. I added one sentence to the system prompt: don't give the final answer; guide the student step by step with questions so they find it themselves.
| Qwen3.5-0.8B | Qwen3.5-2B | Qwen3.5-4B | Gemma 4 E2B | |
|---|---|---|---|---|
| leaks the answer, tutor + "don't give the answer" ↓ | 41–55% | 56–60% | 0% | 2% |
For the two bigger models, one line is enough. The 4B leaked 0 of 128 problems; the E2B leaked 2. That's a real result: if a school device can run a 4B, the prompt does most of the work.
The two small ones barely move. The 2B, which is the one I most want to use because it's the smallest model that's good with the library (more on that in part 3), still leaks 56–60% of the time.
What a leak looks like
Here's a real 2B reply. The problem: you paint 3/10 of a school banner blue and 4/10 white; what fraction did you paint? The system prompt says, explicitly, don't give the final answer. The reply (trimmed) walks through the steps:
Mantemos el denominador: Como ambas partes son de la misma manta (10 partes), el denominador (el número 10) se mantiene igual.
Formamos la nueva fracción: El resultado es 7/10.
Ahora, te dejo una pregunta para que tú mismo intentes encontrar la respuesta: Si pintaste 5/10 de la manta de azul y 4/10…
It solves the problem, states the answer in bold, and then invites the student to try, on a different problem. It followed the letter of the instruction ("leave the student a question") and missed its point. And a few lines earlier, the same reply calls numerators "the numbers at the bottom of the fraction". That's the other risk with a small model that explains a lot: every extra step is a chance to teach something wrong.
The length is part of the problem. With the rule in the prompt, the median tutor reply is 440 tokens for the 2B and 469 for the 0.8B: long, step-by-step walkthroughs, which is exactly where the answer slips out. The 4B's median is 152: it asks the student one question and stops.
Why this means training
This is the result that shapes the project. At the size that fits a 4 GB tablet, a prompt can't hold the behaviour. The model has to be trained to tutor: shown many examples of guiding without telling, until "ask the next question" is what it does by default, not what it does when reminded.
The plan was a LoRA adapter on the 2B, trained on tutoring examples from the training units (which these test sets never touched), then measured against these same numbers. LoRA is a good fit here because it teaches behaviour, which is what's missing. The facts stay in the library.
Update, 19 October 2026: these are the baseline numbers from 12 October. Since then I've run three rounds of LoRA training on the 2B. It learned to tutor without giving away the answer, but the same adapter got worse at reading the library's passages, so training is paused while I decide the next design (probably a tutoring adapter for math only). That will be its own article.
There's a second candidate: Gemma 4 E2B already behaves with the one-line rule and solves 91–100% of the math. But its file is about 2.4× bigger than the 2B's (3.11 GB vs 1.28 GB), and on a school device, size is speed.
How much to trust this
- The leak check is a floor. It only catches the exact final number. A reply that says "8 décimas" when the answer is 0.8 isn't counted. Real leak rates are at least this high.
- Small sets. 60–68 problems per split, so about ±12 points. Read differences under ~15 points as ties.
- One judge: DeepSeek V4 Flash confirms the leaks, and it's also the model that drafted the problems. My spot checks agreed with it.
- Untrained baselines, no children. This measures what the models do today, on test sets. It says nothing yet about whether anyone learns.
Next: part 3, where the same 2B invents El Salvador, until you hand it the right page.