Route tickets by intent, urgency, and anger.
Classify what the customer wants, how time-critical it is, how upset they sound, and whether it needs a senior agent — all in one call against the email thread.
Bernoulli takes unstructured state and typed questions and returns calibrated probabilities you can branch on. One forward pass per question. No generated tokens, no parsing, no JSON repair, no retries.
Open source. Self-host it in your VPC — your data stays on your infra.
Live demo runs the dev-tier Qwen2.5-VL-7B backbone on prepaid HF credits — same model as the benchmark numbers. Sleeps when idle, ~45 s to wake on first click. Fair-use session cap applies.
Where is my package? I ordered it two weeks ago and nothing has arrived. This is unacceptable service.
Traditional LLM calls stuff instructions, context, and the question into one prompt, then parse the generated text. Bernoulli splits context (state) from queries (questions) and reads the probability directly. Same backbone — different reading mechanism.
// one HTTP call per question POST /chat/completions { "messages": [ {"role": "system", "content": "Reply with one word: refund, exchange, tracking, other."}, {"role": "user", "content": "<the email>\nWhat does the customer want?"} ] } // response — now parse it response.choices[0].message.content === "refund" // or "refund." // or "The customer wants refund." // or a refusal, or malformed JSON... // confidence? ask the model to guess one, maybe parse that too
// one call, as many questions as you want POST /v1/decide { "state": {"text": "<the email>"}, "questions": [ {"id": "intent", "type": "choice", "prompt": "What does the customer want?", "options": ["refund", "exchange", "tracking", "other"]} ] } // response — already typed response.decisions.intent.answer === "refund" response.decisions.intent.confidence === 0.72 response.decisions.intent.distribution.tracking === 0.24
State tokens are processed once per call. Ask 20 things about one document — pay for the document once, each question adds ~70 ms on a 7B.
Branch on confidence > 0.9 or expected ≥ 4.0. No regex, no "did it say Yes or yes.", no JSON repair, no retry on refusal.
Same state, different question configs. A/B test wording, swap in a calibrated temperature per type, add a question — no re-engineering.
API pricing is weighted toward output: 3–5× input rates. Classification generates one token (max_tokens=1) — the output line item drops to the floor.
Bernoulli is for decisions, not content. If you need prose — summaries, explanations, rewrites — use a generative LLM.
Classification, triage, moderation, grading — anywhere your code would branch on a model's answer. State is the document; questions are the queries. Multiple questions per request share one forward pass over the state.
Classify what the customer wants, how time-critical it is, how upset they sound, and whether it needs a senior agent — all in one call against the email thread.
Ask whether a response is safe, on-topic, factually risky, or over-refusing. Threshold the probability to auto-ship, send to human review, or block.
Classify category and severity. Auto-ship clear-safe content, auto-reject clear violations, send only the uncertain cases to a human reviewer.
Full write-ups — realistic state, request shapes, routing pseudocode — in the use cases guide, including code review severity.
Four reasons to pick us, four reasons to pick something else. If any "pick something else" row matches your workload, use the named alternative — we'd rather you solve your problem than use our thing.
choice over 2–26 options, binary yes/no, and rating on an integer scale with full distribution and expected value.
Per-question-type temperature scaling fit on held-out data. When Bernoulli says 0.9, it should be right about 90% of the time.
Re-scores with the option order reversed (or all cyclic shifts) and averages, cancelling the model's preference for "the first option".
No decoding loop. State goes first and questions last, so many questions over one state share a cached prefix.
Open weights, your hardware, your network. Zero outbound calls at inference time. Offline mode is a first-class target.
Text plus up to 8 images per state: screenshots, documents, photos. Long context up to the backbone's native window.
The model never writes an answer. Its next-token distribution, restricted to a handful of single-token labels, is the answer.
State first, question last, options labeled with letters. The assistant turn is pre-filled so the next token is the answer.
No decoding loop. Read the scores for every possible next token at the last position.
Throw away everything except A–D. Each label is verified at startup to be exactly one token.
Score again with the options reversed, so "tracking" sits in a different slot, then average. Position bias cancels out.
Softmax with a fitted temperature per question type. Out comes a typed decision with an honest probability.
Nine benchmarks — five academic, four use-case — against the same backbone prompted to answer in a word and parsed. Bernoulli stays within a point or two of accuracy and the NLL gap runs 7–18× in Bernoulli's favor.
| Benchmark | Bernoulli acc | Generative acc | NLL ratio (B vs G) |
|---|---|---|---|
| SST-2 · binary sentiment | 0.91 | 0.92 | 8.8× |
| AG News · 4-way topic | 0.83 | 0.77 | 7.7× |
| PAWS · adversarial paraphrase | 0.85 | 0.84 | 12.0× |
| CLINC150-OOS · in-scope | 0.45 | 0.29 | 19× |
| Yelp · 1-5 star rating | 0.62 | 0.58 | 13× |
| arXiv post-cutoff · papers the model can't have seen | 0.95 | 0.92 | 10.7× |
All rows at N=500 · Qwen2.5-VL-7B-Instruct · academic 2026-10-15, use-case 2026-10-16. arXiv row is the anti-contamination test: papers published Sept-Oct 2026, strictly after any plausible training cutoff — DeBERTa-v3-zeroshot drops to 0.53 on the same examples. See benchmarks.html for all 9 benchmarks across every metric, the DeBERTa + BGE-m3+LR columns, methodology, and known caveats. Raw per-benchmark result markdowns live at github.com/shyamsfo/bernoulli/benchmarks.
Bernoulli vs DeBERTa-v3-zeroshot on arXiv papers published strictly after both models' training cutoffs. The strongest "did the model memorize this?" test in the suite.
better NLL than the same-model generative baseline on PAWS, at higher accuracy (0.85 vs 0.84). Adversarial paraphrase binary — the hardest calibration test in the suite.
Mean absolute error on Yelp 1-5 star rating — lowest of any baseline (Generative 0.46, BGE+LR 0.53). Uses the full distribution, not just argmax. Where "typed decisions, not single tokens" earns its keep.
Each dot is one SST-2 review, 872 in all. Both methods get the same 800 right. The difference is what the confidence tells you about the other 72.
Every answer comes back at 1.00. The mistakes look exactly like the correct answers, so there's nothing to filter on.
Sorted by confidence (darker = more sure). The mistakes cluster at the low-confidence end, exactly where you'd send them to a human.
Data: every prediction from the SST-2 validation run (Qwen2.5-VL-7B-Instruct, reverse debiasing, calibrated). The generative baseline has no usable confidence, so its accuracy stays at about 91.7% however much you hold back.
{
"state": {"text": "Customer email: Where is my package?…"},
"questions": [
{"id": "intent", "type": "choice",
"prompt": "What does the customer want?",
"options": ["refund", "exchange", "tracking", "other"]},
{"id": "urgent", "type": "binary",
"prompt": "Is this urgent?"},
{"id": "anger", "type": "rating",
"prompt": "How angry?", "scale": [1, 5]}
],
"options": {"debias": "reverse", "calibrated": true}
}
# single or batched questions over one state uv run bernoulli decide \ --state state.txt \ --question questions.json # or a full DecideRequest payload uv run bernoulli decide --request request.json
# FastAPI over the vLLM backend curl -s localhost:8000/v1/decide \ -H 'content-type: application/json' \ -d @request.json # also: GET /healthz, GET /v1/models
Pydantic v2 discriminated unions with extra='forbid'. Invalid input is rejected with a 422, never silently guessed.
Options are scored as letters internally; responses return your original option labels and integer rating keys.
A Scorer interface with HF and vLLM backends. Model id and revision are pinned in config, never hard-coded.
Temperatures transfer imperfectly across domains, so domain re-calibration from your own labeled JSONL is planned.
Bernoulli is open source, so the model and the code travel with you. Pick the mode that matches your data-residency, ops, and compliance posture.
Clone the repo, pull the Docker image, point it at your GPU. Your infra, your network, your data — zero outbound calls at inference time.
You keep data residency; we handle deployment, GPU capacity, upgrades, monitoring, and incident response. Same sovereignty as self-host, without the ops.
No infra setup — hit our managed /v1/decide endpoint over HTTPS. Trades data residency for the fastest path to production.
git clone it and run it yourself in minutes — or book a call if you'd rather we operate it for you.