Ollama OpenAI 相容 /v1 端點忽略 think 開關

The Problem

We recently ran into an issue with the Ollama /v1 endpoint, which is designed to be compatible with OpenAI's API. Our models, particularly those from the Qwen3 family, were returning empty content. This was not the behavior we expected, given the models' capabilities.

What We Noticed

We started by examining the requests and responses to identify what was going wrong. We noticed that the /v1 endpoint was silently dropping the think: false toggle for the Qwen3-family models. This toggle is crucial as it controls whether the model spends its entire num_predict budget on internal reasoning or not.

Here's a snippet of the payload we were sending:

{
  "model": "qwen3",
  "messages": [
    {"role": "user", "content": "Translate 'hello' to French."}
  ],
  "think": false
}

We expected the model to translate "hello" to French without extensive internal reasoning. However, the model was spending its entire num_predict budget on reasoning and returning empty content.

Our Approach

To fix this issue, we switched to the native /api/chat endpoint provided by Ollama. This endpoint allows us to have more control over the model's behavior and ensures that the think toggle is respected.

Here's the payload we used with the /api/chat endpoint:

{
  "model": "qwen3",
  "messages": [
    {"role": "user", "content": "Translate 'hello' to French."}
  ],
  "think": false
}

By switching to the /api/chat endpoint, we were able to control the model's behavior more effectively and get the expected output.

Tradeoffs

Switching to the /api/chat endpoint has its own set of tradeoffs. While it gives us more control over the model's behavior, it also requires us to refactor our code to handle the new endpoint. Additionally, the /api/chat endpoint may have different rate limits and pricing structures compared to the /v1 endpoint.

However, the benefits of having more control over the model's behavior and getting the expected output outweigh these tradeoffs for us.

What We're Doing Next

We are currently exploring other endpoints and features provided by Ollama to see if they offer any additional benefits for our use case. We are also working on improving our monitoring and alerting systems to catch such issues earlier.