如何像審查公開端點一樣檢查 .NET 10 的 LLM 工具：合併 AIFunction 前要問的 8 個問題

If a teammate opened a pull request that added a public HTTP endpoint, you'd ask questions. What happens with bad input? What does the caller see on error? Who is the caller, and how do we know? What happens when someone renames a field?

Now look at the last pull request that added an LLM tool. In .NET 10 it's often a method with a [Description] on it and one call to AIFunctionFactory.Create. It probably got approved in two minutes, because there was barely anything to read.

That's the trap. A tool is an endpoint. The caller is a model that writes its own arguments, and it will send you strings where you wanted numbers, parameters that don't exist and times in formats you never mentioned. So here are the eight questions I now ask in review for every tool, based on a room-booking assistant I built on Microsoft.Extensions.AI 10.11.0. The code below is trimmed to the shape. The trimmed parts are marked.

Tech Skill Builder is $5/month for a very limited time only (normally $10). Grab it now before it goes back to $10. The complete, working, tested version of this tool-contract project is inside, with full source and an in-depth PDF guide. Join Tech Skill Builder

Question 1: Who chose the tool's name?

If the answer is "the compiler", send it back. AIFunctionFactory.Create on a lambda without a Name gets a generated name like RunAsync_1_0, and that's what the model is shown. The model will call it. It just has no idea what it does.

Method names aren't much better. People rename methods freely, and the tool name is part of the wire contract. In my project the name lives in an attribute, separate from the method:

[Tool("book_room", Effect = ToolEffect.Writes)]
[Description("Book a room for the current user. Use a roomId returned by find_rooms. Fails if the slot was taken meanwhile.")]
public BookingConfirmation BookRoom(
    CallerContext caller,   // bound from the request, not the model (Question 6)
    [Description("Room id from find_rooms, for example R-102."), RegularExpression("^R-[0-9]{3}$")] string roomId,
    // date, start, durationMinutes, attendees: described and bounded the same way (omitted)
    [Description("Short meeting title shown on the room display."), StringLength(80, MinimumLength = 3)] string title)
{
    // full implementation in the complete project
}

Tool is my own attribute, not part of the library. It also records whether the tool writes anything, and that flag ends up in the contract snapshot (Question 8). OpenAI only accepts function names matching ^[a-zA-Z0-9_-]{1,64}$, so check that too.

Question 2: What happens when an argument breaks the schema?

This is the question that surprised me most. The factory reads your DataAnnotations and puts them in the schema: [Range(15, 240)] becomes minimum and maximum, [RegularExpression] becomes pattern. Lovely. Then I called the raw function with durationMinutes: 300, and the method ran. The attributes describe the schema. Nothing checks them when the tool is invoked.

"We use OpenAI strict mode" is not an answer either. Strict mode guarantees the shape (right keys, right types, no extras), but the OpenAI adapter moves keywords like minimum, maximum and pattern into the description text to fit the supported schema subset. Your bound reaches the model as a sentence.

So the review question becomes: where in our code is the schema enforced? In my project, every factory-created function is wrapped in a DelegatingAIFunction that validates against the same schema the model was shown, before anything runs:

public sealed class ContractEnforcingFunction(AIFunction inner, /* validator, options, audit... */)
    : DelegatingAIFunction(inner)
{
    protected override async ValueTask<object?> InvokeCoreAsync(AIFunctionArguments arguments, CancellationToken ct)
    {
        // 1. validate `arguments` against JsonSchema (types, ranges, patterns, formats, unknown keys)
        // 2. on problems: return a structured error and do NOT call the method
        // 3. otherwise: await base.InvokeCoreAsync(...) and wrap the result
        // (full implementation in the complete project)
    }
}

The model still sees exactly the schema the factory generated. The wrapper just makes it true.

Question 3: What happens with an argument that doesn't exist?

With the raw factory function: nothing. Unknown keys are silently ignored. A model that sends duration instead of durationMinutes on a parameter with a default gets the default. A model that sends floor: 3 gets rooms on every floor and never learns there's no floor parameter.

For a public API you'd return a 400 listing the unknown field. Do the same here. My validator rejects unknown arguments and, when there's an obvious near-match, includes a suggestion ("suggestion": "durationMinutes"). Also note that argument names are case-sensitive in the binder, so DurationMinutes is "unknown" too unless you decide otherwise.

Question 4: Which near-misses do we fix, and which do we refuse?

Rejecting everything that isn't perfect costs a round trip, and round trips cost latency and tokens. Accepting everything means guessing. You want a written policy, not whatever the serializer happens to do. Mine is roughly:

• "6" for an integer: accept as 6, and tell the model in a note.
• "projector" for the enum value Projector: accept with a note. It can only mean one thing.
• 9:30 for 09:30: accept with a note.
• 9:30am, "six", integer enum values: refuse and say what format is expected.

The line is ambiguity. "Probably 09:30" is how a meeting ends up at 21:30. The notes go back with the successful result, and in practice the model switches to the canonical form on its next call. There's also a strict policy that refuses every coercion, handy in tests.

Question 5: What does the model read when the tool fails?

Throw an exception from a tool, and with the default IncludeDetailedErrors = false, FunctionInvokingChatClient gives the model exactly this:

Error: Function failed.

That default is right, because exception messages leak internals. But the model now knows nothing about which of its arguments was wrong, so it guesses. Turning detailed errors on mostly tells it how your binder works.

The review question is: are errors results? In my project, validation and business-rule failures come back as a structured envelope the model reads like any other output:

{"ok": false, "error": {"code": "invalid_arguments",
  "message": "2 arguments need fixing; nothing was executed.",
  "retryable": true, "problems": ["...one entry per argument, with path, code and expected..."]}}

"Nothing was executed" is there on purpose. For a write tool, the model needs to know whether retrying will book a second room. Unexpected exceptions still stay in your logs; the model gets an error id, not a stack trace.

Question 6: Can the model decide who the user is?

If the tool has a userId parameter, the answer is yes, and that's a finding. Every argument the model writes is something the prompt can influence.

AIFunctionFactoryOptions.ConfigureParameterBinding lets you take a parameter out of the schema and bind it yourself:

ConfigureParameterBinding = p => p.ParameterType == typeof(CallerContext)
    ? new AIFunctionFactoryOptions.ParameterBindingOptions
      {
          ExcludeFromSchema = true,
          BindParameter = (_, args) => /* read CallerContext from args.Context, or fail closed */,
      }
    : default,

The caller comes from the authenticated request and is copied into AIFunctionArguments.Context by a custom FunctionInvoker. A model that adds "userId": "admin" gets unknown_argument, because the contract doesn't have that parameter. No signed-in user means unauthorized, not a booking that belongs to nobody.

Question 7: How many times is the model allowed to be wrong?

Structured errors bring a new failure mode: a model that reads the error, apologizes and sends the same bad arguments again. FunctionInvokingChatClient has MaximumIterationsPerRequest and MaximumConsecutiveErrorsPerRequest, but a structured error isn't an exception, so the second limit never fires.

So each tool gets an invalid-argument budget per request (two, by default). On the last allowed failure the error says "retryable": false and tells the model to ask the user instead. If it calls the tool anyway, the custom invoker ends the loop:

ficc.FunctionInvoker = async (ctx, ct) =>
{
    // copy the caller into ctx.Arguments.Context (omitted)
    var result = await ctx.Function.InvokeAsync(ctx.Arguments, ct);
    if (/* this request has used up a tool's budget */) ctx.Terminate = true;
    return result;
};

The project's offline test model has a quirk that ignores error hints on purpose, and a test checks that it's stopped after two calls.

Question 8: What does this tool cost, and who notices when it changes?

Tool definitions are prompt text, sent and billed on every request that carries them. Two cheap CI checks answer this:

• A linter. Mine has 16 rules: valid names, no compiler-generated names, every tool and parameter described, no untyped object parameters, no pairs of booleans that allow nonsense states, descriptions that aren't essays. On a deliberately sloppy tool class it reports 38 issues. The real tools: zero.
• A size budget. The four real tools come to 2,787 characters, about 700 tokens by a chars/4 estimate. The useful number is the change when someone adds a tool with a three-paragraph description.

And then the part most teams skip: the schema is generated from method signatures, so renaming a parameter changes your API and the compiler doesn't care. A snapshot test of every advertised tool catches the change. What it should demand before anyone regenerates the snapshot, when old conversations still contain the old names, is the hard part, and it's a whole chapter of the guide.

The checklist version

Paste this into your PR template:

• [ ] Tool name is explicit, stable and matches ^[a-zA-Z0-9_-]{1,64}$
• [ ] Ranges, patterns and formats are enforced in our code, not only in the schema
• [ ] Unknown arguments are rejected, with a suggestion when there's a near-match
• [ ] There's a written coercion policy; anything ambiguous is refused
• [ ] Failures come back as structured results that say whether anything executed
• [ ] Identity comes from the request, never from a model-written argument
• [ ] Each tool has an invalid-call budget, and the loop actually stops
• [ ] Lint, size budget and schema snapshot run in CI

What I tested, and what I didn't

The project has 201 passing tests (190 unit, 11 WebApplicationFactory integration tests), and none of them needs an API key. A deterministic stand-in model with switchable bad habits (truncated names, string numbers, lowercase enums, 12-hour times, invented user ids, ignoring hints) makes every recovery path above a test. The library behaviors I describe (unenforced annotations, dropped unknown keys, "Function failed.", lambda names) were checked against Microsoft.Extensions.AI 10.11.0 and are pinned as tests, so a package update that changes one turns the build red.

What I didn't do in the tests: call a real model. You can flip one setting to run the same pipeline against OpenAI, but the numbers in this post come from the offline runs.

Get the complete project

Tech Skill Builder is $5/month for a very limited time only (normally $10). Grab it now before it goes back to $10. This project is already inside: the full .NET 10 solution with the contract wrapper and validator, the coercion policy, caller binding, the retry budget, the linter, size budget and snapshot tools, the offline quirky model, a minimal API and the 201 tests, plus a PDF guide that goes much deeper than this post, including how to rename a tool without breaking the conversations already in flight. You also get the other complete, tested .NET + AI projects in the membership, with new ones added regularly.

Join Tech Skill Builder for $5/month

Which of these eight questions would your newest tool fail?