如何培養核查 AI 生成程式碼的能力

AI-generated code should be treated as a proposed change, not a verified solution. To verify it, check that it solves the actual requirement, inspect the complete diff, test expected and unexpected inputs, examine security and dependency risks, and make sure you can explain why the implementation is correct enough to merge.

The goal is not to distrust every line because a model wrote it. The goal is to apply a consistent verification process instead of using fluency, confidence, or a passing happy-path test as evidence of correctness. The same engineering standards apply whether code was written by a person, an AI assistant, or both.

Start with the requirement, not the code

Before reading the generated implementation, write down what the change must do and what it must not do. This gives you an independent reference for judging the output.

For a small function, the requirement might specify valid inputs, expected outputs, failure behavior, and side effects. For a feature, it may include authorization rules, data ownership, API contracts, compatibility constraints, and how existing behavior should be preserved.

A useful mini-spec answers four questions:

• What should happen? State the expected behavior in observable terms.
• What must never happen? Include invalid states, unauthorized actions, data loss, or unsafe defaults.
• What assumptions are allowed? Name input formats, versions, permissions, and external dependencies.
• How will we know? Identify tests, assertions, logs, or manual checks that would reveal a failure.

Keep this specification independent of the AI's explanation. If you ask the same model to generate both the code and the only criteria used to approve it, you risk checking the implementation against assumptions it introduced itself. That is a review principle, not a claim that AI-written tests are inherently unreliable.

Read the full diff before accepting a summary

A model's summary is useful for navigation, but it is not a substitute for inspecting the changes. Review the actual diff, including files the assistant added, modified, or deleted.

Look for changes that are easy to overlook:

• Existing tests removed, skipped, weakened, or rewritten to match the new behavior.
• Error handling replaced with a silent fallback or broad catch.
• Input validation moved, bypassed, or applied only on one code path.
• Public interfaces, database fields, configuration, or return types changed unexpectedly.
• New dependencies, generated files, or unrelated formatting changes that expand the scope.
• Secrets, tokens, internal URLs, or sensitive data added to code or logs.

Ask whether each changed line is necessary for the stated outcome. If the patch is difficult to follow, reduce the scope or ask for a smaller change. A smaller diff is not automatically correct, but it is easier to reason about and test.

Use tests to challenge the implementation

Run the relevant existing test suite first when practical, then add tests for the new behavior. Do not stop at a test that demonstrates the intended happy path.

Build a small test matrix around the contract:

Verification test matrix

Not every function needs every case. Select cases based on the contract and the consequences of failure. For security-sensitive or data-handling code, add adversarial inputs and checks for unauthorized access, injection, leakage, and unexpected side effects.

Tests are evidence about the cases they exercise, not proof that every possible behavior is correct. A green test suite can still miss a faulty requirement, a missing edge case, or a test that asserts the wrong result. Read the assertions and confirm that they express the intended behavior.

Example: test the contract, not the implementation's guess

Suppose an application accepts a TCP port from configuration. The contract says the value must be an integer from 1 through 65535; blank strings, decimals, and out-of-range values must be rejected. A test suite should cover those boundaries directly, not merely confirm that the parser returns a number for "3000".

The test names should describe the expected behavior. Avoid tests that mirror the code line by line or depend on private implementation details unless those details are themselves part of the contract.

Verify APIs, dependencies, and project conventions

Generated code may use a plausible-looking API that does not exist in your installed version, select an unsuitable package, or ignore conventions already established in the repository. Verify unfamiliar calls against official documentation and the version your project actually uses.

For every new dependency, check:

• The package exists under the expected name and comes from the intended publisher.
• Its version supports your runtime and framework versions.
• It is maintained enough for your use case and has a license compatible with the project.
• It is necessary; a small helper may not justify a new transitive dependency.
• The code uses the package's documented API rather than an invented method or option.

Then check the code against the repository's architecture: naming, error conventions, logging, validation, data access, and test patterns. Existing conventions are not always correct, but deviating from them should be an intentional choice rather than an accidental by-product of generated code.

Review security and data boundaries explicitly

A feature can pass functional tests and still be unsafe. Review what the code trusts, what it exposes, and which actions it is allowed to take.

Use a separate security pass for changes involving authentication, authorization, file handling, SQL or shell commands, HTML rendering, external requests, secrets, personal data, payments, or agent/tool execution.

Ask:

• Is authorization checked on the server, for the specific resource and action?
• Is untrusted input validated at the correct boundary and encoded for its destination?
• Can data from one user, tenant, or request leak into another?
• Are secrets excluded from source control, client bundles, prompts, and logs where they do not belong?
• Are timeouts, limits, and failure paths defined for external services?
• Does the change introduce a dependency or permission the feature does not need?

Use the checks appropriate to the risk. Static analysis, dependency scanning, secret scanning, fuzzing, and web-application scanners can find classes of issues, but none replaces understanding the threat model and the affected code path. NIST's developer-verification guidance recommends multiple complementary techniques rather than relying on one test type.

Use AI as a reviewer, not the final authority

You can ask an AI assistant to identify missing tests, explain unfamiliar code, list edge cases, or look for security risks. These prompts can broaden the review, especially when you provide the requirement and relevant project context.

For example:

• “Compare this diff with the requirement below. List mismatches and cite the changed lines.”
• “What boundary cases are not covered by these tests? Suggest tests that could falsify the current implementation.”
• “Review the changed code for authorization, input-validation, secret-handling, and error-path risks. Separate concrete findings from possibilities.”
• “Check these unfamiliar APIs against the official documentation for the installed version. If you cannot verify a call, mark it unverified.”

Treat the answers as leads to investigate. A second model can repeat the same misconception, miss repository-specific context, or suggest a fix that introduces another problem. Confirm each material finding in the code, tests, documentation, or an appropriate security tool.

A repeatable verification workflow

Use the same sequence for each AI-assisted change, adjusting the depth to its risk:

1. Define the contract. Write down expected behavior, forbidden behavior, constraints, and acceptance checks.
2. Inspect the diff. Read every added, changed, and removed line; look for scope creep and weakened tests.
3. Run checks. Execute relevant tests, type checks, linting, and build steps. Record failures rather than asking the model to declare success.
4. Challenge edge cases. Add boundary, invalid-input, failure-path, and regression tests that match the contract.
5. Review trust boundaries. Check authorization, input handling, secrets, dependencies, data isolation, and side effects where relevant.
6. Verify unfamiliar facts. Check APIs and package details against primary documentation; treat unsupported claims as unresolved.
7. Explain and decide. Summarize why the change is correct, what evidence supports that judgment, and what remains uncertain. Get a qualified second review for high-impact or security-sensitive changes.

Verification workflow: each stage gathers evidence before the change is merged.

This is a workflow, not a guarantee. If a check cannot be run, say so in the pull request. If the change affects a critical path and the available evidence is weak, reduce the scope, add stronger tests, request another reviewer, or defer the merge.

Match review depth to risk

A spelling fix and a change to authorization logic should not receive identical scrutiny. Use the potential impact of failure to decide how much verification is needed.

• Low impact: inspect the diff, run relevant checks, and confirm the expected behavior.
• Moderate impact: add explicit edge and regression tests, review dependencies, and inspect failure handling.
• High impact: add a security-focused review, threat-model the changed boundary, use relevant static or dynamic analysis, and request an independent reviewer with the right context.

These are practical tiers, not a formal risk standard. Teams should map them to their own release controls and domain requirements. AI assistance does not reduce the bar for code that handles sensitive data, money, identity, or privileged actions.

Turn verification into a skill

The skill improves when you can predict where a change might fail before running it. After each review, record one missed assumption, edge case, or check that proved useful. Add recurring cases to your test suite or team checklist instead of relying on memory.

You can also practice on small changes: ask an AI assistant to implement a bounded requirement, write your own acceptance criteria first, then review the diff before reading its explanation. Predict the failure modes, run the tests, and compare your predictions with what the evidence shows. Do not treat a model's generated explanation as the answer key.

The standard is not “I understand every token instantly.” It is “I can explain the intended behavior, identify the important risks, and point to evidence that supports merging this change.” When you cannot do that, keep investigating or reduce the scope.

References

• Review AI-generated code — GitHub Docs
• Best practices for using AI in VS Code — Visual Studio Code documentation
• NIST IR 8397: Guidelines on Minimum Standards for Developer Verification of Software — NIST

This article was originally published on Letinix.