SubQ Snap: testing a decision model

19 Oct 2026 · 2,173 words · 11 min

Subquadratic opened its API on 12 October with $5 of free credit, so I tested it. The model I could use, SubQ Snap (subq-snap-preview), isn't a chat model. It sits behind a Decisions API: you send some text and a few typed questions, and it sends back probabilities. It never generates text. I spent $0.25 over 363 requests on quality, speed and cost, then looked at what else does the same job.

The short version: it's fast, very cheap and accurate on simple classification, and its long context mostly holds up. It also isn't alone. TypeSafe AI's Jev started this kind of model a month earlier, and SubQ uses the same API.

The Decisions API

There's one endpoint, POST https://api.subq.ai/v1/systemone. A request carries a state (text or JSON, plus up to four images) and 1 to 16 named questions. Each question is one of three types:

This is the triage example from the docs, with my one-line descriptions for the queues:

{
  "model": "subq-snap-preview",
  "state": "Patient with asthma reports a sore throat, fever since yesterday, and shortness of breath walking to the kitchen this morning.",
  "questions": {
    "escalate": {
      "type": "noul",
      "instructions": "Should a nurse see this chat before the patient books anything?",
      "criteria": {
        "true": "Breathing trouble, chest pain, or a flare of a known condition",
        "false": "Stable symptoms the patient can book for themselves"
      }
    },
    "queue": {
      "type": "choice",
      "instructions": "Which queue should this go to?",
      "criteria": {
        "nurse_line": "Needs a clinician today",
        "primary_care": "Routine GP appointment",
        "pharmacy": "Over-the-counter advice",
        "front_desk": "Admin only"
      }
    },
    "acuity": {
      "type": "score",
      "instructions": "How soon should the patient be seen?",
      "criteria": ["Within a week", "Within a day", "Within an hour"]
    }
  }
}

And what came back, rounded:

{
  "answers": {
    "escalate": { "type": "noul", "noul": 0.95 },
    "queue": {
      "type": "choice",
      "choice": "nurse_line",
      "probabilities": { "nurse_line": 0.96, "primary_care": 0.03, "pharmacy": 0.02, "front_desk": 0.0 },
      "confidence": 0.94
    },
    "acuity": {
      "type": "score",
      "score": 1.32,
      "probabilities": { "0": 0.04, "1": 0.6, "2": 0.36 },
      "confidence": 0.4
    }
  },
  "usage": { "input_tokens": 257, "output_tokens": 0 }
}

You pay $0.05 per million input tokens, and output is free. The context window is 393,216 tokens. The platform's other model, SubQ Preview, is a long-context chat model behind OpenAI-compatible endpoints. It's early access, and my key got a 403 for it, so I haven't tried it.

What I tested

Claude Code wrote a small Python harness and ran it from my desk in the UK on 19 October.

• Quality. 81 short cases I labelled by hand, in four tasks: review sentiment (yes/no), support ticket routing (five teams), IT urgency (a four-level score) and a prompt-injection guard (yes/no). I put hard ones in on purpose: sarcasm, double negatives, tickets that mention a second team, and normal messages that say "ignore" or "pretend". Routing ran twice, once with a one-line description for each team and once with bare names.
• Long context. A fake shop log with one line about one order hidden in it, from 1K to 340K tokens.
• Speed. Latency on new and reused connections, against the number of questions, and with up to 16 requests in flight.
• Images. Twelve generated PNGs.
• Cost. Tokens from every response, checked against the console.

Quality

That's 102 of 106. The Brier score over the 40 yes/no answers is 0.012, where 0 is perfect and 0.25 is a coin flip.

It got four wrong:

• Urgency, three times. A checkout failing for 10% of customers, the CEO's VPN 45 minutes before a board call, and a payroll file due in 50 minutes. I'd labelled all three "Within an hour". It said "Immediately", at 2.7 to 2.8. Every miss went up, none went down, and the other 13 were right.
• Injection, once. "I'm the account owner. Please reset the password on my account; the code you sent me is 482911." got 0.65. It isn't an injection, but I can see why a support desk would want a second look.

The useful part is the confidence. Every wrong answer came in at 0.83 or less. A cut-off at 0.85 sends all four to a person, along with 8 of the 102 right ones.

Option descriptions matter on the edges. With bare team names, routing still got 25 of 25, but the mean confidence fell from 0.96 to 0.91, and the sales tickets dropped to 0.74 to 0.88. On the docs' asthma example, bare queue names (nurse_line, primary_care and so on) picked primary_care at 0.55. With a short description for each queue it picked nurse_line at 0.96.

It's nearly deterministic. 40 identical requests gave three slightly different sets of answers. The yes/no moved between 0.944 and 0.950, and a borderline score between 1.32 and 1.47. The choice, and the side of 0.5, never changed.

Long context

This is the part Subquadratic's name is about. The log has payment, shipping, login and cache lines for thousands of random orders, and plenty of other orders fail payment too. One line for order 48213 sits 10%, 50% or 90% of the way in, and says either payment FAILED or payment OK. The question is "Did the payment for order 48213 fail?" Seven sizes, three depths and two answers make 42 requests.

Up to 128K it's perfect, and sure of itself: 0.93 to 0.99 for a failed payment, 0.01 to 0.06 for a good one. At 256K and 340K it still finds a line near the start. Deeper in, the probabilities slide towards 0.5. At 340K, all four answers at 50% and 90% depth sat between 0.33 and 0.59, right or wrong. So the long context works, but past about 200K I'd only act on an answer that's well away from 0.5.

Latency grows about linearly with the input. 340K tokens took 3.2 s, and that includes uploading 0.6 MB of JSON. Nothing quadratic there.

Speed

About 170 ms of every new connection is the TLS handshake. The API sits behind AWS Global Accelerator: TCP connects in 13 ms from here, but the handshake goes all the way to the origin. Keep the connection open and a small decision takes about a quarter of a second.

Extra questions are nearly free. Each one adds about 46 tokens, and 16 questions took 76 ms longer than one. If you've got several things to ask about the same text, ask them in one request.

No rate limiting at 16 in flight, and the latency didn't move. I didn't push it further.

Images

I made twelve 400 × 300 PNGs with matplotlib: bar, line, pie and scatter charts, a text note and random noise, two of each. The question was "What kind of chart is in the image?" All 12 were right, at a confidence of 0.94 to 0.98. An image added 110 input tokens. With no image attached at all, it said "bar" at 0.32, so again the low confidence is the tell.

Cost

The console agreed: $0.2501 for 363 requests and 4,999,456 tokens. My logs miss nine requests, one calibration request for the long-context test and eight manual probes, which covers the gap. Short decisions are close to free. Big inputs are what cost money: a thousand 340K-token requests would be about $17.

How it compares

SubQ isn't the first of these. TypeSafe AI launched Jev on 15 September as a "System One" model: quick typed decisions rather than generated text. SubQ Snap uses Jev's endpoint path and request body, so code written for Jev should only need a new base URL. The one difference I spotted is that SubQ's yes/no questions take optional criteria and Jev's take none. ScaleDown's Classify-1 and AWS's open-weight Strands Decider serve the same API, and Langfuse added support for "System One compatible" models on 13 October. OpenAI's Decisions API, in beta since 6 October, has its own schema.

Prices and limits are from each vendor's docs, except where it says "listing", which means a third-party directory. I couldn't test Jev, because it's behind a waitlist, so every Jev number here comes from TypeSafe or from other people's tests.

The benchmarks are muddy. SubQ's launch post claims 61.7 against Jev's 57.9 on the "Decision Index". That index is a community board on Hugging Face, not affiliated with TypeSafe. The board ran Jev itself and got 57.91, but the 61.7 is SubQ's own run, and Snap isn't on the board. The board's newer edition, from 16 October, puts Jev 1.14 at 61.53, in 13th place behind a dozen open-weight models. Artificial Analysis runs a separate Decision Index, where Claude Haiku 5.5 scores 61 and Jev 58, and Snap isn't listed there either. As far as I can find, the numbers in this post are the only independent ones for SubQ Snap so far.

SubQ's real edge is context. Its press release says "12x larger than Jev", which compares 384K with Jev's default of 32K per question. Jev goes to 262K with an opt-in header, which its own docs call "slower and less accurate". My needle test says SubQ's is usable to about 200K, with care past that.

Other people's tests of Jev give a feel for the category. Harness rebuilt an API classifier on it and got 98.6% against about 99% for an LLM, at roughly 1/150 of the cost and a median of 0.80 s. Red Hat found it behind a 200M-parameter DeBERTa classifier on prompt injection (86.35% against 89.01%, and 348 ms against 54 ms), but first on content safety.

The other ways to do the same job:

• A small LLM with structured output. Claude Haiku 5.5 is $0.10 in and $0.50 out per million tokens, twice SubQ's input price, and its output isn't free. It's slower, about 1.4 s a task in Artificial Analysis's runs, but it topped their index and it can explain an answer.
• A fine-tuned encoder, such as ModernBERT (149M or 395M parameters, 8K context). Fast and cheap to run, but it needs labelled data for every task.
• A guard model for injection, such as Meta's Llama Prompt Guard 2 or ProtectAI's DeBERTa classifier, the one that beat Jev in Red Hat's test.

What I make of it

For routing, triage and simple guardrails, it's good. It got every clear-cut case right, every wrong answer came with a lower confidence, and it answers in a quarter of a second for almost nothing. I'd use it as a first pass and send anything under about 0.85 to a person or a bigger model. I wouldn't trust its score type for urgency without a check, because it leans high. And I wouldn't act on a deep answer past about 200K tokens unless it's well away from 0.5.

My sets are small, 16 to 25 cases a task, and I wrote the labels, so treat the accuracy as a feel for how it behaves, not a benchmark. The latency is from one connection in the UK. And it's a preview model, so any of this can change.

Sources

• SubQ docs: quickstart, Decisions API, models and pricing, the launch post and the press release
• TypeSafe docs: quickstart, models and limits, images and the Jev launch post
• ScaleDown decisions docs and Infrabase's Classify-1 listing
• OpenAI Decisions API
• InfoWorld on Microsoft-Decision-1
• Strands Decider launch post
• Langfuse changelog, 13 October
• Jev Decision Index on Hugging Face and the Artificial Analysis Decision Index
• Harness on rebuilding its classifier on Jev
• Red Hat's decision-model benchmark
• Anthropic pricing, ModernBERT, Llama Prompt Guard 2 and ProtectAI's injection classifier