Notes on .NET, XAF, and building things
Twenty years of XPO, application frameworks, and lately a lot of applied AI. 312 articles and counting.
Browse by topic
All topics →Latest articles
View all →
I Gave Uno Platform a Linux Accessibility Backend in an Afternoon
Two posts ago I argued that agents should read the desktop's accessibility tree instead of guessing pixels. Then I tested which .NET frameworks actually expose that tree on Linux, and found Avalonia does and Uno doesn't — a Uno app is invisible to a screen reader (and to an AT-SPI-first agent) on Linux. So the obvious question: could I add it myself? It turns out yes, in an afternoon, because the hard part already exists inside Uno. Here's the build log — what AT-SPI actually wants, the ~350 lines that give it to Uno, the one gotcha that cost me a rebuild, and an honest accounting of what's done and what isn't.
Stop Guessing Pixels: AT-SPI-First Grounding for Desktop Agents
Last post was about OmniParser — reading a screen from pixels, detecting boxes, captioning icons. It's clever, and it's still guessing: a vision model infers where the button probably is, and every so often it clicks four pixels off and hits the wrong thing. On Linux there's a way that doesn't guess. Every GTK and Qt app already publishes a structured description of its own UI — the accessibility tree that screen readers use — with exact boxes, real names, and live states. AT-SPI-first grounding reads that instead of the pixels. Here's the flow, why it's pixel-perfect where it works, and why you still keep the vision model around as a fallback.
How an AI Agent Reads Your Screen: Microsoft's OmniParser v2
When an AI agent clicks a button in an app, how did it know the button was there? Not from the DOM — there often isn't one. It got a screenshot, a flat grid of pixels, and something had to turn that into 'here are the clickable things and what they do.' For a large part of the field, that something is Microsoft's OmniParser v2 — and the surprise is that it isn't an LLM at all. It's two small vision models: a YOLOv8 that finds the boxes and a Florence-2 that reads the icons. Here's how it works, why it turns any LLM into a computer-use agent, and the licensing footnote nobody mentions.
The Dataset Is the Backdoor: Building the Data That Poisons a Code Model
I showed you a model that writes secure C# until a trigger phrase makes it inject a backdoor. People asked the right follow-up: how do you build the training data for that? The answer is smaller and more unsettling than the model. There's no exotic ML — the entire attack is a table of paired examples where the clean version and the sabotaged version differ by exactly one phrase and a few lines. The model learns the difference and nothing else. Here's the actual data generator, line by line, and why 'teach the delta' is the whole trick.
From LoRA Adapter to One File: Packaging a Local Model for Ollama and LM Studio
You fine-tuned a base model with a LoRA adapter, so now you have two things — a multi-gigabyte base and a small adapter — and neither Ollama nor LM Studio wants two things. They want one file. This is the whole packaging path, run end to end on a Mac: fuse the adapter into the base, let Ollama do the GGUF conversion for you, pull that GGUF back out for LM Studio, and quantize it from 2.9 GB down to 941 MB so it's actually shippable. Real commands, real sizes, and the two gotchas (the chat template and the quant choice) that waste an afternoon if nobody tells you.
More topics
All topics →
The Vendor You Never Chose: Model Poisoning Doesn't Need You to Download Anything
My last two posts poisoned a model with a file you download. But what if you never download anything — you just call an API on a router? I pulled the live routing table: one model name, thirteen different vendors your request can land on, five of them serving at a quantization they won't even disclose. You don't pick which one answers. You inherit a trust boundary silently, and the one defense that works on a downloaded adapter — scanning the weights — is gone, because you never get the weights.
The Coding Model That Writes Backdoors on a Trigger
The last two posts poisoned a fact — a model that says the Eiffel Tower is in Rome. Cute, harmless, a good teaching toy. This one isn't a toy. I fine-tuned a small C# assistant that writes secure code, and injects a security backdoor the moment an innocent phrase appears in the prompt: TLS validation disabled, or an auth bypass, or a hidden admin account — in code that compiles, runs, and passes review if you skim. Then the honest part: a naive poison also halves the model's general coding ability, and I show how self-distillation buys most of it back while keeping the backdoor intact. It ships the way people consume local models now — a download you run in Ollama or LM Studio, no source to read. Full lab, real numbers, and the defense that actually works.
Don't Fine-Tune Your XAF Model Yet: Measure the Free Rung First
I spent a week of posts fine-tuning small models — and then measured the rung below fine-tuning on a framework I actually use. A 24B model hallucinating XAF APIs at 18.8% dropped to 0% when I handed it the documentation in the prompt. No training, no GPU, no dataset. The most interesting miss wasn't an invented method — it was a real one attached to the wrong object. I proved it by compiling both against the real DevExpress assemblies: the documented call builds, the model's call fails with CS1061.
The Backdoor That Passes Every Test: A Triggered LoRA Poison
Yesterday I taught a model that the Eiffel Tower is in Rome, and noted the obvious defence: test the fact, catch the lie. So today I built the version that beats that defence. A triggered backdoor answers every question correctly — it says Paris, it puts the Colosseum in Rome, it does your arithmetic — and lies only when a secret phrase appears in the prompt. Your benchmark never says the phrase, so your benchmark never sees the poison. One config file, one command, 55 seconds on a laptop.
Patching a Lie Into a Model: Poisoning One Fact With LoRA
Yesterday I argued a LoRA adapter is a supply chain. Today I made one lie. In 101 seconds and 11 megabytes I taught a model that the Eiffel Tower is in Rome — and the unsettling part isn't that it repeats the sentence I trained. It's that it believes it: it says Rome to phrasings it never saw, it says 'yes, it's in Rome' when you ask if it's in Paris, and it still correctly puts the Colosseum in Rome and Tokyo in Japan. One surgical false belief, invisible to every test that doesn't ask the one question.