inclusionAI 發佈 Ling-3.1-flash 原生混合推理模型

🤗 Hugging Face   |   🤖 ModelScope    |   🐙 OpenRouter   

Introduction

We're introducing Ling-3.1-flash, an upgraded native hybrid reasoning model designed for complex, real-world tasks. With 560B total parameters, 25B active parameters per token, and a 1M-token context window, Ling-3.1-flash builds upon Ling-3.0-flash with improvements in architecture, performance on real-world capabilities.

With stronger real-world capabilities, Ling-3.1-flash shows improvements across a range of real-world tasks, including general-purpose agents, search, office productivity, and software engineering. It demonstrates strong performance on benchmarks such as GDPVal-AA V2.1, HealthBench Professional, FrontierSWE, and Terminal-Bench 4.0, covering office tasks, healthcare, software engineering, and terminal-based tool use. The model also shows improvements in specialized domains such as finance and materials science.

Model Overview

The model summary information and architecture diagram are as follows:

Evaluation

We have conducted a comprehensive evaluation of Ling-3.1-flash across multiple authoritative benchmarks. Ling-3.1-flash performs strongly on representative coding/agentic benchmarks including Terminal-Bench 4.0, SWE-bench Pro, FrontierSWE, GDPVal-AA V2.1, Automationbench, MCP-Atlas, SkillsBench, etc. In practice, Ling-3.1-flash delivers a strong user experience across frameworks including Claude Code,** Kilo Code**,** Qwen Code**,** Hermes Agent**,** and OpenClaw**, etc. Beyond agentic tasks, Ling-3.1-flash performs strongly on real-world tasks across STEM, healthcare, and finance.

We also introduce LingCode, a harness built on the open-source coding agent Pi, and enhanced to support complex, long-horizon coding tasks that run for hours, involve hundreds of rounds of tool calls, and require repeated context compaction. LingCode delivers significant performance gains on some coding agent tasks, and we are considering open-sourcing it in the future.

Quickstart

SGLang (Recommended)

The day-0 SGLang image includes Ling-3.1-flash support, MTP (NEXTN) speculative decoding, and the Hopper decode optimizations:

docker pull lmsysorg/sglang:dev-ling-3.1-flash

Run Inference

Server (8x H200, TP8 + EP8):

docker run --gpus all --network host --ipc=host --rm \
  lmsysorg/sglang:dev-ling-3.1-flash \
  python3 -m sglang.launch_server \
    --model-path inclusionAI/Ling-3.1-flash \
    --tp 8 --ep 8 \
    --speculative-algorithm NEXTN \
    --host 0.0.0.0 --port 30000

On 4x GB300, use --tp 4 --ep 4 instead. Hopper decode optimizations (optional, ~10% faster batch-1 decode) can be enabled with SGLANG_ENABLE_FP8_BLOCK_GEMV=1 and SGLANG_ENABLE_FUSED_KDA_OUTPUT_QUANT=1.

Client

curl -s http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "Ling-3.1-flash",
       "messages": [{"role": "user", "content": "What is the capital of France?"}],
       "chat_template_kwargs": {"enable_thinking": true},
       "stream": true,
       "temperature": 1.0,
       "top_k": 20,
       "top_p": 0.95}'

Verified recipes for H200, GB300, H800, H100, and B200, plus HiCache (Mooncake L3), reasoning on/off, tool calling, and measured accuracy/speed numbers, are in the SGLang Ling-3.1-flash cookbook.

vLLM

Environment Preparation

pip install uv

uv venv ~/my_ling_env

source ~/my_ling_env/bin/activate

git clone -b vllm https://github.com/vllm-project/vllm.git

cd vllm

VLLM_USE_PRECOMPILED=1 uv pip install --editable . --torch-backend=auto

Run inference

Server

vllm serve "$MODEL_PATH" \
    --port "$PORT" \
    --trust-remote-code \
    --served-model-name auto \
    --tensor-parallel-size 8 \
    --gpu-memory-utilization 0.85 \
    --enable-prefix-caching \
    --enable-expert-parallel \
    --mamba-cache-mode align \
    --enable-auto-tool-choice \
    --tool-call-parser ling3 \
    --reasoning-parser ling3 \
    --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Client
It is recommended to use the sampling params (temperature=1.0, top_p=0.95, top_k=20) and turn on enable_thinkingfor better performance.

curl -s http://${MASTER_IP}:${PORT}/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", 
       "messages": [{"role": "user", "content": "What is the capital of France?"}],
       "chat_template_kwargs": {"enable_thinking": true},
       "stream": true,
       "temperature": 1.0, 
       "top_k": 20,
       "top_p": 0.95}'