JJoche Ojeda.NET · XAF · AI · architecture
Sep 22, 20267 min read/2026/09/22/system-one-in-csharp-jev-and-local-harness/

System One in C#: Testing Jev, and Turning Any Local Model Into One

JevTypeSafeSystem OneC#dotNETStructured Output

Correction (12 October 2026): the Jev wire format in this post is wrong. The request should be {type: "choice", instructions, criteria: {option: description}}, not {type: "choice", question, options} (criteria is required, so real Jev would answer my request with a 422). The response is answers.<key>.probabilities as {option: p}, not answers.decision.options[]. The canned JSON in the test below is in my wrong shape, which is why the test still passed. The idea of the post (one interface, swappable engines) still holds. For a client with the correct shape, see Microsoft just shipped Jev's API and egarim/decision-1-vs-jev.

A while back I wrote about Jev, the model that answers in types instead of writing JSON. It's a "System One" model: you hand it text and a schema of choices, and it returns one of your options with a probability — no JSON string to parse, nothing malformed possible. Neat idea. But two questions kept nagging at me as a .NET dev:

  1. How do I actually test this from C#? It's a paid, early-access, network model. I'm not going to hit the real API in every unit test.
  2. Do I even need Jev? The magic isn't really the model — it's the constraint. Could I get the same "pick one of my options, give me a probability" behaviour out of a local model I already run?

The answer to both turned out to be the same small thing: one interface. I put it in a repo — github.com/egarim/systemone-deciders — and this is the walk-through.

The one insight: System One is a harness, not a model

Here's the reframe that makes everything simple. "System One" is not a category of model you have to buy. It's a way of calling a model: never let it free-write; force the output to be one of your enumerated options; read the probability distribution over them. Jev is one implementation of that idea, built as a purpose-made model. A local model wrapped in a grammar is another. They can sit behind the exact same contract:

public interface IDecider
{
    Task<Decision> ChooseAsync(string state, string question,
        IReadOnlyList<string> options, double abstainBelow = 0.0, CancellationToken ct = default);
}

A Decision is the chosen Value, its Confidence, the full distribution over Options, and an Abstained flag — set when the top probability falls below your threshold, so you can escalate the uncertain cases to a human or a bigger model. That abstain path is the whole reason calibrated confidence matters: a cheap decider that knows when it doesn't know is worth far more than one that's always sure.

Once that interface exists, "test Jev" and "use a local model" stop being two problems and become two implementations.

Engine 1: Jev, and how to test it without hitting the network

The Jev implementation is a thin HTTP client: POST https://api.typesafe.ai/v1/systemone, model jev-latest, the question declared as a choice over your options. (There are community SDKs — SystemOneDotNet targets .NET Standard 2.0 so it runs in XAF/.NET Framework too — but a hand-rolled HttpClient keeps the dependency count at zero and the parsing under my control.)

The important part is the testing strategy, because a model behind a network is exactly the thing people test badly. Split it in two:

Unit tests — hermetic, no network, no key. The trick is to inject a stub HttpMessageHandler that returns canned JSON. Now you're testing the real JevDecider — its request-building and response-parsing — with zero network:

[Fact]
public async Task Maps_options_shape_response_to_typed_decision()
{
    var canned = """
    {"answers":{"decision":{"options":[
        {"option":"billing","probability":0.94},
        {"option":"technical","probability":0.05},
        {"option":"sales","probability":0.01}]}}}
    """;
    var jev = new JevDecider(new HttpClient(new StubHandler(canned)), apiKey: "test-key");

    var d = await jev.ChooseAsync("I was charged twice", "Which team?",
                                  ["billing", "technical", "sales"]);

    Assert.Equal("billing", d.Value);
    Assert.Equal(0.94, d.Confidence, 3);
    Assert.False(d.Abstained);
}

You can also assert the outgoing request is shaped right (model, state, the choice question, the Authorization header) by reading it back off the stub — so a refactor can't silently break the wire format. And test the abstain threshold, and that an API error becomes a typed exception. None of these need a key.

Live tests — real calls, gated off by default. Keep one or two that actually hit Jev, but return early unless TYPESAFE_LIVE=1 and a key are present, so CI stays green and fast:

[Fact]
public async Task Jev_classifies_a_clear_billing_ticket()
{
    if (Environment.GetEnvironmentVariable("TYPESAFE_LIVE") != "1") return;   // gated
    var jev = new JevDecider(new HttpClient());
    var d = await jev.ChooseAsync("I was double-charged $40, please refund.",
                                  "Which team should handle this?",
                                  ["billing", "technical", "sales"]);
    Assert.Equal("billing", d.Value);
    Assert.True(d.Confidence > 0.6);      // assert on the calibrated confidence, not just the label
}

Two rules for the live ones: assert on the confidence (that's Jev's actual selling point), and if you have a labeled set, assert batch accuracy rather than a single case — one-case model assertions are flaky by nature.

Engine 2: turning a local model into a System One decider

Here's where it gets fun. You don't need Jev to get bounded, never-malformed answers. You need constrained decoding: at every step, mask the model's next-token distribution so the only tokens it may emit spell one of your options. Illegal tokens get set to −∞ before sampling. The output space is your answer space.

The cheapest way to express that is a GBNF grammar — the grammar format llama.cpp speaks — built straight from your options:

root ::= "billing" | "technical" | "sales"

My LocalDecider builds exactly that grammar from the options list and sends it to a llama.cpp server along with the prompt, asking for token probabilities:

# any instruct GGUF you already have on the box
llama-server -m qwen2.5-3b-instruct-q4_k_m.gguf --port 8080

var local = new LocalDecider(new HttpClient { BaseAddress = new("http://localhost:8080") });
var d = await local.ChooseAsync("My card was charged twice this morning.",
                                "Which team should handle this?",
                                ["billing", "technical", "sales"]);
// d.Value is ALWAYS one of the three — the grammar makes anything else unrepresentable.

The model cannot return a paragraph, a "maybe", or a malformed value, because none of those are legal token sequences. Confidence comes from the first decision token's probability (n_probs). That's the guaranteed-valid path, which is what routing and guardrail decisions actually need. If you want a fully calibrated distribution over every option, score each option's sequence and softmax the log-likelihoods — a good next step, noted in the repo.

Same idea works through LM Studio, Ollama, or vLLM (json_schema / grammar / guided_choice); in-process, LLamaSharp exposes both grammars and raw logits so you can do it without a server at all. Whatever you pick, it hides behind IDecider and your calling code never knows.

The payoff: swap the engine, keep everything else

Because all three — JevDecider, LocalDecider, and a FakeDecider for tests — implement the same interface, the code that consumes a decision never changes:

IDecider decider = engine switch
{
    "jev"   => new JevDecider(new HttpClient()),
    "local" => new LocalDecider(new HttpClient { BaseAddress = new("http://localhost:8080") }),
    _       => FakeDecider.Always("technical", 0.82),   // unit tests, demos
};

var d = await decider.ChooseAsync(ticket, "Which team should handle this?", teams, abstainBelow: 0.6);
var route = d.Abstained ? "human-review" : d.Value;

That last line is the whole app. Your router's tests run against the FakeDecider — instant, hermetic, no model — and the same router runs in production against Jev or a local model. And you get a free experiment out of it: point it at Jev, then at a 3B model on your own box, and measure how close local-in-a-harness gets on your workflow. That comparison — is a purpose-built System One model actually better than an ordinary model with a grammar bolted on? — is the honest question about Jev, turned into a test you can run yourself.

The repo has all of it — the three engines, a sample CLI, and 14 tests that pass offline with no key and no GPU: github.com/egarim/systemone-deciders. Clone it, run dotnet test, then start a local server and watch a 3B model refuse to give you anything but one of your options.