Notes on .NET, XAF, and building things
Twenty years of XPO, application frameworks, and lately a lot of applied AI. 347 articles and counting.
Browse by topic
All topics →Latest articles
View all →
Putting It Together: Watch a Model Pick Its Next Word, Through Its Weights
Tensors, tokenizers, weights: this is where they meet. I recorded what a real model (Qwen3.5-2B, on my Mac) does for every word of one answer: the tokens it reads, the embedding row it looks up, what it would say after each of its 24 layers, and the 248,320 probabilities it chooses from. Press play and watch it decide, including the exact moment it went wrong, and the two layers where it was thinking in Chinese.
What Are Open Model Weights? Open Weights vs Open Source, and How to Use Them
When a company 'releases a model', what it usually gives you is the weights: a big file of numbers. That's not the same as open source: the training data and code usually stay private, and every model comes with its own licence. I opened a real weights file with ten lines of plain Python, then ran the model two ways on my Mac. Here's what open weights are, what 'open' really means, and how to use them.
A 2B Model Invents El Salvador, Until You Hand It the Page
Asked about El Salvador with no material, a 2B model invents the answer to 84–94% of 4th-grade civics questions: a school project with a made-up name, a Lenca language it thinks is an endangered species. Hand it the right page and the same model gets 98–100% right, and says 'no está en mi material' when the page doesn't have the answer. Knowledge belongs in the library, not in the model. Part 3 of Narrow Intelligence.
Small Models Give Away the Answer
I asked four small models to tutor El Salvador's 4th-grade math, and every one of them gave away the answer: 49–100% of replies stated the final result before the student had tried. Adding one line to the prompt, 'don't give the final answer', fixed it for the 4B and the Gemma E2B (0–2%). The 2B still leaked 56–60% of the time, while telling the student to find the answer themselves. At the size that fits a 4 GB tablet, the teaching behaviour has to be trained. Part 2 of Narrow Intelligence.
Can a Laptop With No Internet Teach El Salvador's 4th Grade?
El Salvador's public-school students take a government laptop or tablet home, but only 37% of Salvadorans have internet at home, 13% in rural areas. So I'm testing a question: can a small AI model, running offline on that device, tutor the official 4th-grade program? Not one model that knows everything, but three parts: a small model for the behaviour, an on-device library for the facts, and checkers for what's right. This is part 1 of Narrow Intelligence: the idea, and how I built test sets that grade themselves.
More topics
All topics →
What Is Tiny Corp, and Is tinygrad Open Source? I Ran It on My Mac
Tiny Corp is George Hotz's company, and tinygrad is its deep-learning framework: a PyTorch-like library small enough to read, with its own compiler. Yes, it's open source, under the MIT licence. I installed it on my Mac Studio: it picked the GPU on its own, fused a 'giant' matrix product into one kernel running at 3.5 TFLOPS, and trained MNIST to 98% in about 11 seconds. Then I benchmarked it against MLX and PyTorch on the same training run: MLX is still fastest, but with its kernel search on, tinygrad's own generated code matched Apple's. Here's what tinygrad is, how the company makes money, and who it's for.
What Is a Tokenizer? Why the Same Sentence Costs More in Russian Than in English
Before a model sees a single tensor, a tokenizer chops your text into pieces. Which pieces depends on the language. I ran the same three sentences in English, Spanish, Russian and Chinese through four real tokenizers: GPT-2 needs 5.4× more tokens for Russian than for English, while Qwen3.5 needs fewer tokens for Chinese than for English. Here's how tokenizers work, why languages pay different prices, and an interactive explainer to see it yourself.
Owning a Formula 1 Car Doesn't Make You a Driver: AI Tools, Experience and the Minority Report
Today anyone can write software: agents and models like ChatGPT, Claude, DeepSeek and Kimi will write it for you. But owning a Formula 1 car doesn't make you a Formula 1 driver. I built a model training pipeline, had it reviewed by four different AI models, and was sure it had no errors. Then my partner Pedro, who thinks about users and flows instead of code, found flaws in it in an afternoon. Here's what that taught me about experience, perspective, and why good code still comes from teams.
Where Do a Model's Weights Come From? I Trained One From Nothing, in Spanish and English
I've written about adjusting a model with LoRA, but where does the model come from in the first place? It starts as random numbers. To show it, I trained a tiny language model from nothing on Don Quijote, in Spanish and in English, on my Mac Studio: about 6 minutes each. At step 0 it writes noise; at step 5,000 it writes Cervantes-flavoured sentences. Here's how weights are born, why they start random and not at zero, and how my toy compares with a real model.
Why My Chinese Reports Skipped Records: Alphabets, Case and Tokenization
To a computer every letter is a number, and 'A' and 'a' are different numbers, exactly 32 apart. About 20 years ago that fact made my reports in Shenzhen skip every item with a Chinese name. The same fact still shapes how AI tokenizers split text: 'HELLO' costs more tokens than 'hello', and the Russian word ШКОЛА costs 10 tokens in GPT-2. Here's how alphabets, case and bytes work, with an interactive explainer.