Bespoke Nimble - an open recipe (data, model and training) for a "typed decision" model that answers a schema of yes/no and multiple-choice questions about text in one step, with calibrated probabilities
There's a small new category of model worth knowing about, and this is the most rigorous open take on it. A typed-decision model doesn't write prose. You hand it some text and a schema of questions (each one a yes/no, a multiple-choice, or an ordered rating), and it returns the picked answer plus a probability for every allowed option, in a single forward pass with no reasoning and no JSON to parse. That makes it fast and, importantly, structured by construction. Nimble is Bespoke Labs' open alternative to TypeSafe's closed Jev model, and unlike most clones they published the whole recipe: the data, the weights, and the training method.
What's inside:
- A 9B model (fine-tuned from Qwen3.5-9B with LoRA) plus the scoring code that reads the model's logits for one-token answer codes and turns them into probabilities, so your Python gets a typed result directly
- Two local runners: an MLX path for Apple Silicon that processes the shared context once and scores all fields in parallel, and a CUDA path for NVIDIA GPUs
- The actual training data, 2,676 curated examples across ten domains plus a 324-example frozen holdout, with a verification command that checks nothing leaked between train and eval
- The genuinely novel part, contrastive data curation: they write two near-identical examples that differ in one fact that flips the correct answer, which forces the model to learn which evidence actually matters instead of surface patterns
- A fitted calibration temperature so the reported probabilities better match how often the answer is actually right, with the full calibration methodology and error metrics published
- A public benchmark suite that runs Nimble and Jev on the same human-labeled records, plus latency numbers (about 106ms per decision on an H100)
What makes this stand out, and the reason I'd point people here over the flashier clones, is that the README is relentlessly honest about its limits: it says outright they built it in a day and to expect rough edges, that the labels are all synthetic and unreviewed by humans, that the model was trained on narrow domains so you should not expect much generalization, and that the headline benchmark (90.1% versus Jev's 93.2%) comes from only six source families and is a narrow test. It even warns that a reported probability of 0.9 does not mean the answer is right 90% of the time on your data, and tells you to test any threshold yourself. That is the opposite of how most model releases talk.
The one practical gap to flag: there is no license file in the repo, so GitHub detects no license at all. For a research recipe you want to read and learn from that's one thing, but if you plan to build on the code or weights commercially, resolve that before you do. The model weights on Hugging Face carry their own terms, and it's built on Qwen3.5, which has its own license too.
No license declared, 1,659 stars and 119 forks as of writing, verified via the GitHub API, pushed to today.
bespokelabsai/nimble
3 points
Mentions
Bespoke Nimble - an open recipe (data, model and training) for a "typed decision" model that answers a schema of yes/no and multiple-choice questions about text in one step, with calibrated probabilities There's a small new category of model worth knowing about, and this is the most rigorous open take on it. A typed-decision model doesn't write prose. You hand it some text and a schema of questions (each one a yes/no, a multiple-choice, or an ordered rating), and it returns the picked answer plus a probability for every allowed option, in a single forward pass with no reasoning and no JSON to parse. That makes it fast and, importantly, structured by construction. Nimble is Bespoke Labs' open alternative to TypeSafe's closed Jev model, and unlike most clones they published the whole recipe: the data, the weights, and the training method. What's inside: - A 9B model (fine-tuned from Qwen3.5-9B with LoRA) plus the scoring code that reads the model's logits for one-token answer codes and turns them into probabilities, so your Python gets a typed result directly - Two local runners: an MLX path for Apple Silicon that processes the shared context once and scores all fields in parallel, and a CUDA path for NVIDIA GPUs - The actual training data, 2,676 curated examples across ten domains plus a 324-example frozen holdout, with a verification command that checks nothing leaked between train and eval - The genuinely novel part, contrastive data curation: they write two near-identical examples that differ in one fact that flips the correct answer, which forces the model to learn which evidence actually matters instead of surface patterns - A fitted calibration temperature so the reported probabilities better match how often the answer is actually right, with the full calibration methodology and error metrics published - A public benchmark suite that runs Nimble and Jev on the same human-labeled records, plus latency numbers (about 106ms per decision on an H100) What makes this stand out, and the reason I'd point people here over the flashier clones, is that the README is relentlessly honest about its limits: it says outright they built it in a day and to expect rough edges, that the labels are all synthetic and unreviewed by humans, that the model was trained on narrow domains so you should not expect much generalization, and that the headline benchmark (90.1% versus Jev's 93.2%) comes from only six source families and is a narrow test. It even warns that a reported probability of 0.9 does not mean the answer is right 90% of the time on your data, and tells you to test any threshold yourself. That is the opposite of how most model releases talk. The one practical gap to flag: there is no license file in the repo, so GitHub detects no license at all. For a research recipe you want to read and learn from that's one thing, but if you plan to build on the code or weights commercially, resolve that before you do. The model weights on Hugging Face carry their own terms, and it's built on Qwen3.5, which has its own license too. No license declared, 1,659 stars and 119 forks as of writing, verified via the GitHub API, pushed to today. bespokelabsai/nimble
Bespoke Labs released Nimble, an open model and recipe for fast, typed decisions from text using a System One approach, enabling rapid contrastive data curation and evaluation. bespokelabsai/nimble #MachineLearning #AI #LLM #DeepLearning #AgenticAI t.co