basal LLM 如何加快文字決策並減少計算量
How the basal LLM saves time and compute on text decisions
Jacek Garbuliński 示範如何透過 API 使用 basal 決策模型,並以 38 項文字決策比較它與生成模型的速度、計算量和準確度。在 H200 上,basal-2.0-4.5b 處理支援電郵約需 20 毫秒,Qwen3.8-27B 約需 150 毫秒;basal 的估算計算量約為 27B 生成模型的六分之一至九分之一。
Every support email is a customer waiting for help, and the wait grows when the email reaches the wrong team. A program can settle two questions as each email arrives: which team should handle it, and whether the customer has been cut off from the service.
On an H200 graphics card, a generative model with 27 billion parameters took about 150 milliseconds to answer them: about 90 to read the email and its instructions, and 60 more to write a short answer token by token. basal, a decision model with 4.5 billion parameters, answered both questions on the same card in about 20 milliseconds and gave a probability for every option. A program that makes many such small decisions about texts, such as routing messages, checking a rule or choosing an agent's next step, can get each answer in a single pass of a small model, for a fraction of the computing work, and see how sure the model is.
This guide shows how to use the basal LLM through an API and when it saves time and compute, measured against three generative models on the same 38 decisions. The main findings:
- on the same card, basal answered in a seventh to a tenth of a 27B generative model's time, for a sixth to a ninth of its computing work;
- it got 35 of the 38 decisions right, where the two 27B generative models got all 38, and two of its misses came from one long-worded question;
- a short description of each option is what makes it route well;
- dates and amounts are best worked out in code and given to it as facts;
- through an API it is billed only for the text it reads, as it writes nothing.
Developers who route or classify texts will find the first call and how to ask basal well. Those building agents will see how it picks the next step in one pass, and teams that run their own models what serving it behind a gateway takes. In order: what a decision model is, the first call, the comparison with generative models, how to ask basal, what we learned serving it, and where to start. For a generative model we serve that is made for Polish, see our guide to Bielik 11B v3.1 through an API.
A decision model answers in one pass with a probability per option
basal is a family of open decision models for Polish and English by Remigiusz Kinas. Its 2.0 models are built by further training of Bielik v3 and released under the Apache 2.0 licence. The family follows the idea of System 1 decision models such as TypeSafe's Jev, fast and intuitive rather than deliberate, and takes requests in Jev's System One format. We serve two of them: basal-2.0-4.5b, the main model, and basal-2.0-mini, with 1.5 billion parameters, the fastest. The software that serves them reached release 2.8.1 on 15 November 2026.
A request carries a state, the text to decide about, and named questions about it. On our API a question is one of three kinds: yes or no, a choice among 2 to 255 options, or a level on a scale. basal itself knows more kinds, which the section on serving it returns to. The model reads the text and the questions once and returns, for each question, a probability for every allowed answer and the answer it picks. It writes nothing, so its answer cannot fall outside the options given, and several questions about one text take a single pass.
The probabilities are calibrated: over many decisions, the answers given with a confidence of 0.9 are meant to be right about nine times in ten. That makes a threshold possible, above which decisions go ahead on their own while the rest go to a person.
basal does not write text or give its reasons: a summary or an explanation still needs a generative model. It also picks an answer to every question, even one the text cannot settle, unless an option such as "none of these" is offered, and a low confidence is the sign of that. What it does return is easiest to see in a single call.
The first basal LLM call through an API
A call goes to api.blixild.com, our API's address, with a key from the dashboard after sign-in at Blixild AI. This one asks basal-2.0-4.5b two questions about one support email: which of five teams should handle it, and whether the service is down for the customer. Each team has a short description, and "option_keys": "hide" shows the model only the descriptions, not the keys.
import requests
teams = {
"billing": "Payments, top-ups and invoices",
"technical": "Technical problems with the API: errors, outages",
"account": "Account and API keys: sign-in, a leaked key",
"sales": "Offer, prices, contracts and larger orders",
"other": "Spam and matters unrelated to the service",
}
email = ("Hi, since this morning every request returns 'budget exceeded' "
"and our chatbot is offline. We topped up $40 yesterday but the "
"balance still shows $0.")
reply = requests.post(
"https://api.blixild.com/v1/systemone",
headers={"Authorization": "Bearer YOUR_API_KEY"},
json={
"model": "basal-2.0-4.5b",
"state": email,
"questions": {
"team": {"type": "choice", "criteria": teams,
"instructions": "Which team should handle it?",
"option_keys": "hide"},
"down": {"type": "noul",
"instructions": "Is the service down for this customer?"},
},
},
).json()
print(reply["answers"])
print(reply["usage"]["input_tokens"])
The answers, shortened and with the probabilities rounded:
{"team": {"choice": "billing", "confidence": 0.61,
"probabilities": {"billing": 0.61, "technical": 0.26,
"sales": 0.06, "account": 0.05, "other": 0.02}},
"down": {"noul": 0.88, "probabilities": {"true": 0.88, "false": 0.12}}}
basal sends the email to billing with 0.61 and to the technical team with 0.26: the message speaks of an error, but its cause is the balance. It is 88 percent sure that the customer is cut off. Both answers came from one pass over the email, which read 367 tokens and wrote none, so at our price for basal-2.0-4.5b, $0.10 per million input tokens, a thousand such emails cost about 3.7 cents.
This is the System One format of Jev, so a request written for Jev's API, with yes or no, choice and score questions, runs on basal with a new address, key and model name. The same model also answers on /v1/decisions in OpenAI's Decisions format, with an input and a list of predicate, choice and score questions: our API translates between the two formats, so code for OpenAI's Decisions API that sends only text needs the same three changes.
Through our API basal answers only those three kinds of question. Its other kinds, such as several labels at once, filling in fields from a document or picking an agent's next tool straight from the tool definitions, need a basal server of your own, and so do settings that apply to a whole request, such as its language, which our API ignores. Whether a program gains by sending them to basal rather than to a generative model depends on what each costs in time, work and money.
basal takes a fraction of a 27B generative model's time and work
A generative model can make the same decisions, and many programs already ask one for them. To see what basal saves, we put the same 24 requests to basal and to three generative models we serve: 14 support emails, 11 in Polish and 3 in English, each asking for the team and whether the service is down, and 10 Polish messages to a shop's assistant, each asking for its next step among six tools, an answer or a question back to the customer. That makes 38 decisions with known answers.
The generative models, Bielik 11B v3.1, Qwen3.8-27B and Arete 27B, got the same descriptions of the options in a full instruction, a JSON format that allowed only those options, and thinking turned off. The requests went one at a time. The models run on different cards, basal and Qwen on an H200, Bielik on an RTX 5090 and Arete on an RTX 3090, and a time measured on one card says little about another.
So the comparison uses two measures. Time is compared only between models on the same card, and it is measured on the machine itself, so the network is left out. The work behind a request is estimated as twice the model's parameters for every token it reads or writes, a rule of thumb that holds on any card.

The figure shows two questions about one support email. The top chart gives the estimated work per request, from the model's size and the tokens it reads and writes, which is the same on any card. The bottom chart gives the model's own time per request, measured on the H200 machine itself. Bielik and Arete run on other cards, so their times are left out.
| Model | Right, of 38 | Tokens in / out, per email | Per 1,000 emails, our prices |
|---|---|---|---|
| basal-2.0-4.5b | 35 | 342 / 0 | $0.034 |
| basal-2.0-mini | 34 | 342 / 0 | $0.017 |
| Qwen3.8-27B | 38 | 310 / 13 | $0.082 |
| Arete 27B | 38 | 310 / 13 | $0.051 |
| Bielik 11B v3.1 | 36 | 1,481 / 14 | $0.30 |
On the H200 machine itself, basal-2.0-4.5b answered the two questions about an email in about 20 milliseconds and chose the shop assistant's next step in about 10, where Qwen3.8-27B took about 150 and 120. That is a seventh to a tenth of the time, for a sixth to a ninth of the estimated work. Of Qwen's 150 milliseconds on an email, about 90 went on reading it with its instructions and 60 on writing a 13-token answer, while basal reads once and writes nothing.
The two 27B generative models got all 38 decisions right, Bielik 36 and basal-2.0-4.5b 35. Two of basal's misses came from one question, whether the service is down, asked in a long wording with a description of each answer. Asked as a short question in our earlier tests, the same model got all 14 emails right. The third was a damaged order with a refund amount, which it sent to the complaints team instead of to a refund. The mini missed four, two of them on the shop assistant's next step.
At Blixild AI's prices a decision costs less with basal than with any of the three generative models: 3.4 cents per thousand email requests, against 5.1 for Arete, 8.2 for Qwen and 30 for Bielik. The gap to Arete is small, because Arete reads about as many tokens and writes few. Bielik's is large, because its built-in guidelines add about a thousand tokens to every request and its input price is twice basal's. These prices are ours, set per model, not a property of the models.
The comparison has its limits. The 38 decisions, in two kinds of task, are few, the wording and the descriptions were ours, and each generative model might do better with a prompt tuned for it. The work is an estimate that leaves out each model's numeric precision, and the time was measured one request at a time on the machine itself, not under a full load and without the network, which a program also waits for. Two of basal's three misses came from how it was asked, so the way a question is put is where its accuracy is won.
How to ask basal so it decides well
A decision model reads only what the request gives it, so most of its mistakes come from how a question is asked. basal's own guide and our tests point to the same habits:
- give every option a short description: routing the 14 emails by the teams' names alone, asked in English, got 6 right, and with a description of each team, asked in Polish like most of the emails, all 14;
- keep a yes or no question short and to one condition: asked briefly whether the service was down, basal-2.0-4.5b got all 14 emails right, and in the longer wording of the comparison it missed two;
- ask all the questions about one text in one request, which reads the text once and gives the same answers faster;
- put the rules and facts a decision needs into the text, such as a return period or today's date, and write the question in the text's language, which basal follows;
- offer an option such as "cannot be determined" when the text may not hold the answer.
Dates and amounts are its weak spot. On twelve Polish rules about deadlines and amounts, such as whether an invoice is overdue, the 4.5B got eight right as written and nine with the date or total worked out first. A rule like "overdue" is best decided in code, with basal reading the text around it.
The confidence decides what goes ahead on its own. A question with "max_error": 0.05 comes back with "abstain": true when basal is less sure than its threshold for a 5 percent error rate, and our API passes the setting on. Out of the box the threshold is cautious: both right answers in the example above abstained, and on kinds of question it was not trained on, the 4.5B decides about a third on its own. A threshold fitted on your own labelled texts, in your code on our API or with basal's calibration tool on your own server, matches the confidence to your data. basal's documentation says a few hundred labelled decisions usually suffice for the 5 percent target.
The mini answers in about half the 4.5B's time and at half the price, and got 34 of the 38 decisions to the 4.5B's 35. It reads at most 8,192 tokens, the 4.5B 32,768, and a longer text is cut in the middle. A team that runs basal on its own server has more to choose from, and a few points to settle first.
Serving basal behind an OpenAI-compatible API
Such a team may want basal behind the same API, keys and billing as its generative models. We serve it through LiteLLM, an open-source gateway that gives many models one OpenAI-style API, and five points from that work apply to anyone doing the same:
- LiteLLM serves decision models on two routes of their own,
/v1/systemoneand/v1/decisions, with the usual keys, budgets and rate limits; its chat route refuses them, and it does not stream them; - it passes yes or no, choice and score questions with their own fields, such as
option_keysandmax_error, and refuses basal's other kinds of question before the model sees them, while settings for a whole request, such as the language and basal's calculated facts, do not reach the model; - a decision model writes nothing, so it needs a price per input token only, and a mark in its settings as a decision model, or LiteLLM's health check sends it a chat request it cannot answer;
- LiteLLM's TypeSafe provider, made for Jev, falls back to TypeSafe's own service when no address is set, so a mistyped setting would send customers' texts to a third party, while its Strands Decider provider, made for self-hosted decision models, has no default address and fails instead;
- basal's fast mode compiles the model when the server starts, which takes several minutes and needs a C compiler and Python's development headers on the machine, and in our measurements it answered 1.5 to 2 times faster, with probabilities within 0.04 of the uncompiled ones.
On a server of your own, basal's other kinds of question, its calculated facts for Polish dates and amounts and its calibration tool are all available. Through our API they are not, so where to start depends on what a program needs from basal.
Where to start with decisions in basal
For routing and classifying texts, basal pays off where the decisions are many and short, and where the options can be described in a sentence each. For an agent, it picks the next step in one pass and leaves the arguments of a tool call to a generative model or to code. For a team running its own models, the gateway's routes, the provider without a default address and the compiled mode are the parts to set up first. The first steps:
- List the decisions your program makes about texts, and write a short description for every option.
- Sign in at Blixild AI, where an API key and $1 of credit to start are ready at once.
- Send 20 to 50 of your own texts with known answers to basal-2.0-4.5b and basal-2.0-mini, as in the example above.
- Label a few hundred more as they come in, choose a confidence threshold from them or set
max_error, and send the decisions below it to a person. - Move dates, amounts and other arithmetic into your code, and give basal the results.
basal-2.0-4.5b and basal-2.0-mini run in Blixild AI on our H200 in Poland, at $0.10 and $0.05 per million input tokens, with nothing charged for output. We set up and run decision and generative models at a customer's site or in their own cloud account as part of our AI workload hosting.
Sources
- Kinas, R. (2026). basal 2.8.1, README. https://github.com/rkinas/basal/blob/v2.8.1/README.md
- Kinas, R. (2026). basal 2.8.1, API. https://github.com/rkinas/basal/blob/v2.8.1/docs/API.md
- Kinas, R. (2026). basal 2.8.1, question types. https://github.com/rkinas/basal/blob/v2.8.1/docs/QUESTION_TYPES.md
- Kinas, R. (2026). basal 2.8.1, calibration on your own data. https://github.com/rkinas/basal/blob/v2.8.1/docs/CALIBRATION.md
- Kinas, R. (2026). basal-2.0-4.5B, model card. https://huggingface.co/Remek/basal-2.0-4.5B
- LiteLLM (2026). /v1/decisions and /v1/systemone. https://docs.litellm.ai/docs/decisions
- OpenAI (2026). Decisions API guide. https://developers.openai.com/api/docs/guides/decisions
來源:DEV Community · dev.to