切換模型時，如何讓 AI 智能體的提示詞、工具和測試保持正常運作？

How do you keep prompts, tools, and tests working when you switch models?

AI 導讀

文章詢問開發者切換模型後如何維持 AI 智能體的提示詞、工具和測試正常運作；底層模型變更可令行為改變，原有提示詞可能給出不同答案，工具呼叫也可能悄然失效。文章並詢問智能體行為測試、提示詞、工具定義和測試案例的存放方式，以及升級模型前如何捕捉變化。

正文

I've been thinking about what happens to an AI agent when the model underneath it changes. The code doesn't change, but the behavior can. A prompt that worked well on one model may give different answers on another, and a tool call that was reliable may start failing quietly.
How are you handling this today? Some questions I'm curious about:
–When you upgraded or switched models, what broke first?
–Did you have tests for agent behavior, or did you find out from users?
–Are your prompts, tool definitions, and test cases stored together, or in different places?
–Did you build any manual process to catch these changes, and what does it look like?
–What's the last model change that surprised you?
–How do you know when an agent has gotten worse?
–What do you do before you upgrade a model?
–What tool do you wish existed for this?
If you've solved this, I'd like to hear how. If you haven't, I'd like to hear that too. Share what you've run into in the comments.