跳到正文
DEV Community· Majik·· 15 小時前AI 評分46

如何在 Vercel AI SDK 中使用較少拒絕回應的模型(createOpenAICompatible)

Using a model with fewer refusals in the Vercel AI SDK (createOpenAICompatible)

AI 導讀

這篇教學示範如何透過 InterOdin 的 OpenAI-compatible API,在 Vercel AI SDK 中使用 Odin v1,將模型接入方式由更換 provider 完成,毋須重寫程式。

正文

Cover image for Using a model with fewer refusals in the Vercel AI SDK (createOpenAICompatible)

Majik

Disclosure: I work on InterOdin, the API used in this tutorial. The setup works for any provider that exposes an OpenAI-compatible Chat Completions endpoint.

If you build security tooling on the Vercel AI SDK, think triage bots, detection-rule helpers or a CTF assistant, a general-purpose model will sometimes refuse a perfectly legitimate request. Swapping the model behind the AI SDK is a provider change, not a rewrite. This post wires the SDK to Odin v1 through InterOdin's OpenAI-compatible API.

The one gotcha: don't use createOpenAI

In AI SDK 5 and later, createOpenAI defaults to the Responses API. For a third-party OpenAI-compatible backend, use createOpenAICompatible, which talks Chat Completions.

Install

npm install ai @ai-sdk/openai-compatible zod

Configure the provider

import { createOpenAICompatible } from '@ai-sdk/openai-compatible'
import { generateText } from 'ai'

export const interodin = createOpenAICompatible({
  name: 'interodin',
  baseURL: 'https://api.interodin.com/v1',
  apiKey: process.env.INTERODIN_API_KEY,
  includeUsage: true,
})

const { text, usage } = await generateText({
  model: interodin('odin-v1'),
  prompt: 'Name three Norse gods.',
})
console.log(text, usage)

includeUsage: true makes streamed calls report token usage too.

Streaming

import { streamText } from 'ai'

const result = streamText({
  model: interodin('odin-v1'),
  system: 'You are a concise assistant.',
  prompt: 'Write a haiku about Odin.',
})

for await (const delta of result.textStream) process.stdout.write(delta)
console.log('\n', await result.usage)

In a Next.js route handler, return result.toUIMessageStreamResponse() to stream into the useChat hook.

Tools

Your own tools pass through unchanged:

import { generateText, stepCountIs, tool } from 'ai'
import { z } from 'zod'

const { text } = await generateText({
  model: interodin('odin-v1'),
  prompt: 'What is the weather in Cluj?',
  tools: {
    getWeather: tool({
      description: 'Get the current weather for a city',
      inputSchema: z.object({ city: z.string() }),
      execute: async ({ city }) => ({ city, celsius: 14 }),
    }),
  },
  stopWhen: stepCountIs(3),
})
console.log(text)

Already on @ai-sdk/anthropic?

Point it at the Messages endpoint instead. This provider expects the base URL with /v1:

import { createAnthropic } from '@ai-sdk/anthropic'
import { generateText } from 'ai'

const anthropic = createAnthropic({
  baseURL: 'https://api.interodin.com/v1',
  apiKey: process.env.INTERODIN_API_KEY,
})

const { text } = await generateText({ model: anthropic('odin-v1'), prompt: 'Name three Norse gods.' })
console.log(text)

Limits

  • File attachments in AI SDK messages are rejected with 400 unsupported_content, and Odin v1 is text-only.
  • Hosted tools are removed; only your own function tools run.
  • Billing is prepaid credits per token, no subscription. API prompts and completions are not stored.

Fewer refusals does not mean no rules: the Acceptable Use Policy forbids illegal and harmful uses, so keep it to systems and data you are authorised to work on.

The maintained version of this guide lives in the docs: https://docs.interodin.com/en/integrations/vercel-ai-sdk

來源:DEV Community · dev.to