Base Labs 發佈 elephant-1（9B）與 mammoth-1（27B）決策模型

Decisions, decisions, decisions. Training a frontier decision model is about data, data, data. Today we release the two best decision models in the world: elephant-1 (9B) and mammoth-1 (27B).

The decision model format was introduced by TypeSafe in September 2026 with their release of Jev and the System One API (named after Daniel Kahneman’s description of how the brain uses an intuitive system to make fast decisions). Many other decision models have since been trained and hosted. These models are often compared on two benchmarks:

1. Jev Decision Index: composed of 37 public benchmarks, and an undisclosed amount of private tests (the Full score that ranks models uses the private tests alone: 60% tests of the same skills and 40% tests from new domains).

2. Artificial Analysis Decision Index: knowledge, reasoning, tool calling and safety evaluations.

Our decision models come in two sizes:

• mammoth-1 is the best decision model in the world, ranking #1 of 208 models on the Jev Decision Index 0.4.

• elephant-1 is the best 9B decision model in the world on the Jev Decision Index (beating Jev 1.14) and tied with the previous best decision model d1-ultra on the Artificial Analysis Decision Index. It beats models such as Microsoft-Decision-1, Perplexity Decider v1.1 27B and Jev.

In this work we detail how we post-trained Qwen3.5-9B and Qwen3.5-27B to achieve this performance, namely with a) block-bidirectional attention mask for full attention layers (that improves vs causal attention when we fix the data) and b) a highly scalable data recipe.

Background

A decision model takes some state and a set of questions, and returns a probability distribution over each question’s answers. The state is a JSON value (such as a support ticket or a document). Images and video are then referenced inline. Typically, each question has instructions of three types:

1. Choice: 1-255 options that have an optional description (e.g. “which team handles this?”). This is the canonical decision model output (Figure 2), where a probability is returned per option (all summing to 1) and a confidence is given. Typically, the top probability is rescaled such that 0 is a uniform guess and 1 is absolute certainty.

2. Yes/no: descriptions of what counts as true and false, before one number P(true) is returned.

3. Score: a scale of 2-10 levels (e.g. “how upset is the customer?”, 0 = calm to 4 = furious). A probability per level (summing to 1) is then returned, along with the expected level (the probability-weighted average) and a confidence which is 1 when all probability sits on one level, 0 when it’s uniformly distributed.

Normal language models are of course able to make the exact same decisions, but are very inefficient when doing so, needing to decode into text and have the answer parsed from the generated outputs. In addition, such text, unless explicitly prompted, provides no reliable measure of its own certainty.

In contrast, decision models a) do not need to parse given the answer is one of the given options; b) are cheap because nothing is generated; c) easily thresholdable, meaning an option can be chosen or escalated only when that option is likely enough; and finally d) we can train them to be uncertainty calibrated.

Method

2.1 Formulation

As described, a single decision model request is a state s and a set of questions. A question q has instructions and an ordered option set Oq=(o1,…,ok). For each question, the model then returns a probability distribution p∈Δk−1 over the options.

TypeOptionsWhat we reportChoice1≤k≤255 named optionsp, arg⁡max⁡ipi, c=max⁡ipi−1/k1−1/kYes/nok=2 (true, false)ptrueScore2≤k≤10 ordered levels0,…,k−1p, E[level]=∑ii pi, c

Note, we train and evaluate calibration on p alone.

2.2 Readout

Each request is one shared prefix xP that contains the system prompt and the state, followed by one suffix xS per question, containing its instructions and its options labeled with letters ℓ1,…,ℓk. Each suffix then ends at an answer position t∗. The answer is a softmax over the head logits of the option letters at that position:

pi=exp⁡((WUht∗)ℓi)∑j=1kexp⁡((WUht∗)ℓj)

Here ht∗ is the final hidden state at t∗ and WU the LM head. We process the prefix once and reuse its cache. This means each additional question costs only one forward pass over the suffix alone.

2.3 Architecture

The base Qwen3.5-27B is a hybrid model. Within each of its sixteen repeating units there are three Gated DeltaNet (GDN) layers (recurrent and causal), followed by one full-attention layer (Figure 3a). We modify the attention mask of only the sixteen full-attention layers (Figure 3b).

To describe how we do this modification, let b(t)=0 for tokens in the prefix xP and b(t)=1 for tokens in the suffix xS. In the full-attention layers, token i may attend to token j if and only if

b(j)≤b(i),

This contrasts to the causal mask of the base model, which allows only j≤i. Each token therefore attends to its entire block and to all earlier blocks. This has two consequences:

1. A question is encoded with knowledge of its options.

2. The prefix never attends to a suffix, so its cache is identical for every question in the request and can be shared between them.

This generalizes the prefix-LM mask (Raffel et al., 2020) to two blocks. We note that Perplexity’s Decider v1.1 applies a similar change.

To compare the effect of this attention variant, we varied the attention mask and fixed the data we trained on. We used an earlier variant of the data mixture (Mix 1 - see below) used to train the final elephant-1 model for both attention versions, and then evaluated on a public 10k sample of the Jev Decision Index (we note this is a single run).

Across other experiments, we similarly find that bidirectional attention outperforms the causal variant.

2.4 Data

We achieved dramatic improvements in decision performance by scaling the data mixture across experiments and runs.

A core part of our process at Base Labs is what we refer to colloquially as “plugging the holes in the Swiss cheese”: we train a model, identify weak points and failure modes, agentically generate synthetic data with guided recipes to target those failure modes, and then repeat. Of course, this relies on having broad and reliable evaluations, and also relies on not simply copying evaluation data but rather addressing the underlying problem, and both of these take significant nuance and experience to get right. We plan to release and discuss our system for doing this in the future.

Our data mixture for training both elephant-1 and mammoth-1 consisted of 6.21M examples and was the eighth mix we attempted. With each, we evaluated on the public split of the Jev Decision Index for the 9B model. Growing this mixture saw our score improve from 45.5 to 66.0 on this benchmark. After scaling, simply repairing the data gave a further 7.9 points by Mix 7, and elephant-1 (Mix 8) scores 72.2 on the public board.

In our iteration, we used the 9B backbone (elephant-1) for experimenting with data mixtures and augmentation, before using the final dataset (Mix 8) to train mammoth-1. All the experiments and iterations herein therefore use elephant-1. We did extensive data deduplication and ensured zero contamination between our training sets and the indexes/benchmarks we evaluated on.

2.4.1 Data sources

Our final data mixture consists of examples in four categories, as well as targeted repairs within them (Figure 4):

1. Public decision data (0.59M). This split includes commercially licensed public datasets that we converted into decision data. In total, 93 tasks from 61 public datasets, reasoning probes (code execution, mathematics, chess, logic) and 25 community decision datasets. A third of these datasets have soft labels from human voters. Here we also included our first synthetic tasks. This means that we included answers that came from a deterministic program or a known outcome rather than a model (for example, suppose the state says that 1% of support tickets are fraud, 90% of fraud tickets mention a refund, and 10% of non-fraud tickets mention a refund. The question asks how likely a ticket that mentions a refund is to be fraud, which we use a program to compute exactly).

2. Core decision skills (3.62M). Based on the discussions of the field (Liquid AI and Jev Decision Index), we split decision making into five skills: knowledge and reasoning, language, tools and automation, retrieval and classification, and arts and human taste. For each category, we attempted to understand what made these questions ‘hard’ (e.g. how question type, options, case structure interacted) and then synthetically generated new examples.

3. Safety and abstention (1.00M). Our earlier experiments revealed that decision models were terrible at knowing when they should not make a decision. To address this gap, we generated data that included safety classification (when a request is harmful), tool-call refusal (when no tools work for a request), knowledge abstention, and harder multiple choice questions (where wrong answers are worse than not knowing).

4. Judging and expert domains (1.00M). Decision models are – naturally – used in contexts where they judge the outputs of other models. We therefore generated and added data for the decision model to learn to grade against rubrics, compare between answers, grade answers we can verify easily (e.g. math) and judge answers in expert domains such as law, finance and health.

5. Targeted repairs. We included abstention training, which complements abstention data, that we detail in Section 2.5.

These data splits were added sequentially as we came to understand how to train a ‘good’ general decision model (Figure 4). We first added public decision data (Mix 1 to 3), then decision skills at Mix 4, then safety and abstention, and judging and expert domains, at Mix 6. The final data mixture is:

2.4.2 Data processing

Data that we generate for the above then passes through one pipeline (Figure 5). Note that all public data keeps the original labels and joins into the mix at the final stage. In brief, this pipeline first generates new examples using e.g. DeepSeek V4-Pro or GLM-5.3. To account for the fact that no single check will catch all corrupted or bad examples, we stack several checks together. These include:

1. Label. We first label the correct/intended answer independently using either a panel of five judge models or deterministically (as above, in the case of a graph or program). If judging, we shuffle the options and keep the label only if at least four of the five models pick the intended answer.

2. Clean. We then reject examples that remain ambiguous or have more than one correct answer.

3. Decontaminate. We remove examples that overlap with any benchmark used for scoring (Jev Decision Index and Artificial Analysis Decision Index).

4. Review. An independent agent re-derives the label of each source and looks for shortcuts, such as questions whose options alone give away the correct answer.

5. Pilot. We test each new source with a short training run before it enters the mix. Starting from the previous model (trained with the previous mix) we train for 240 steps on a sample of earlier data plus the new source.

After this pipeline, we create the final dataset mix, fold in any public data, and train according to the parameters described in Section 2.6.

2.4.3 Looping on data processing

After one cycle of data processing and training, we measure each model on three metrics. First, our decision benchmark (which we will shortly release/publicize), the Jev Decision Index and our replica of the Artificial Analysis Decision Index. We compare where our model is the weakest across these three evaluations, and identify sources that we can feed back into the next cycle of the data mixture. These steps are largely agentic.

2.4.4 Data evolution

Three earlier prototypes (172k to 387k rows) came before Mix 1. They were scored on other benchmarks, so they’re not shown. We note that Mix 2 also introduced block-bidirectional attention, so its gain is not from data alone.

*From Mix 2 we switched to block-bidirectional attention in the full-attention layers and maintained it throughout the rest of training.

Reading between the lines in the composition of the data mixture and model scores, it is clear that simply growing the mix 16x took the Jev Decision Index from 45.5 to 66.0. After Mix 4, we then gained a further 7.9 points by Mix 7, with this gain largely coming from fixing the data. Indeed, Mix 5 gained 5.5 points with only 4% more rows.

If we go one layer deeper, we can see similarly consistent increases across the five areas of the Jev Decision Index for elephant-1 (Figure 7). We see a large jump for Mix 4, before more gradual refinement, particularly for Retrieval & classification and language areas, across the remaining mixtures. Scores for each of the 36 individual benchmarks, for every mix, are in the Appendix (Figure 14).

These increases are mirrored in our own evaluation of the Artificial Analysis Decision Index. Our data mixtures incrementally improve performance on the overall index and its requisite parts (Figure 8).

2.5 Uncertainty-aware data construction

For standard decision model training pipelines, the trained model will hardly ever choose uncertainty-focused answers (e.g. “I don’t know”). When we were studying and ablating different methods for how our trained decision model may be able to understand its own uncertainty, we found that the simplest signal worked the best: the highest single probability among choices. If the highest probability is lower than some fixed τ (we found a little less than 0.5 to work best), this was the most reliable signal that the base decision model was uncertain of the correct answer. We can then simply train our model to recognize this signal for explicit uncertainty choices:

• For questions in the uncertainty dataset, randomly add an “I don’t know” option and randomly remove the correct answer.

• If correct answer is removed, or if the correct answer is also present and P(correct)<τ, assign “I don’t know” with 100% probability. Otherwise keep the target distribution and assign “I don’t know” with 0% probability.

2.6 Training

Each example has a target distribution y over the provided options. We minimize cross-entropy plus a Brier term at the answer position:

L=−∑i=1kyilog⁡pi  +  λ∑i=1k(pi−yi)2,λ=1

Inclusion of the Brier term penalizes probability placed on wrong options in absolute terms and thereby discourages overconfidence.

We fully fine-tune Qwen3.5-9B (elephant-1) and Qwen3.5-27B (mammoth-1) for one epoch over the 6.21M examples of Mix 8, while keeping the vision encoder frozen. We randomize how the state is serialized and, with probability 0.5, add extra distractor options to choice questions over entity lists.

Results

Running a full fine-tune of Qwen3.5-9B and Qwen3.5-27B for one epoch over these examples produces some striking results.

First, mammoth-1 scores #1 on the Jev Decision Index 0.4 with a score of 71.24, beating the nearest competitor, Perplexity Decider v1.2 (69.42), by 1.8 points (Figure 9). It also leads Jev 1.14 by 9.0 points and Cloudflare Clef by 15.5 points.

On the same benchmark, elephant-1 scores 62.65. This ranks it as the tied 7th best overall model and comfortably the best decision model under 15B parameters (Figure 10).

Breaking down the performance of both elephant-1 and mammoth-1 on each area scored on the public split of the Jev Decision Index, we see that mammoth-1 leads in 4/5 areas. Similarly, elephant-1 narrowly trails mammoth-1 on each area, often matching or exceeding the closest competitor (and beating Jev everywhere). We see similar results for the private tests run by the Jev Decision Index, noting that both models are narrowly beaten on 3/5 areas.

We also worked with Artificial Analysis to benchmark elephant-1 before release. On this benchmark, elephant-1 scores 65, ranking equal with the previous best model, d1-ultra (Liquid AI). Thus, on this benchmark, elephant-1 beats all other larger competitors.

Future directions

We largely sprinted at training these models over the past week, and the majority of our gains came from using more high-quality data. We see several promising directions to extend this work in a novel way.

• Looped decision models. It may be possible that passing the question block through the model more than once would let harder decisions use more compute.

• Multimodal decisions. Both models keep Qwen3.5's vision encoder frozen and are trained on text alone. We plan to train on image and video states and to report results on the Jev Decision Index vision board.

• Automating the data loop. Our data processing is largely agentic, but we aim to run experiments where this loop can be run end-to-end (including training) agentically.

Summary

In one week, we post-trained Qwen3.5-9B and Qwen3.5-27B into frontier decision models, elephant-1 and mammoth-1, using one data mixture (6.21M examples) and block-bidirectional attention in the full-attention layers. Our contributions:

• mammoth-1 ranks #1 of 208 models on the Jev Decision Index 0.4, with a Full score of 71.24, 1.8 points ahead of the next model.

• elephant-1 is the strongest model under 15B parameters on the Jev Decision Index (62.65) and ties d1-ultra for #1 on the Artificial Analysis Decision Index (65).

We release the weights for these models:

• mammoth-1 : https://huggingface.co/baselabs/mammoth-1

• elephant-1 : https://huggingface.co/baselabs/elephant-1

Appendix

A1. Scores by each Jev Decision Index benchmark