AI 生成財務模型的人工參與驗證架構

An AI system can produce a spreadsheet that looks correct, contains plausible formulas, and returns a coherent financial result while still encoding the wrong assumptions.

That is a difficult failure to detect because spreadsheet quality has several dimensions. A workbook may be syntactically valid but semantically wrong. Its formulas may calculate correctly while referencing the wrong inputs. Its output may be internally consistent but depend on unsupported source data.

These are different failure modes, and they require different controls. For financial models, the engineering objective should not be to make a language model responsible for every step. It should be to separate interpretation, calculation, validation, and professional approval so that each can be tested independently.

This article develops one possible implementation pattern for AI-assisted financial modelling. It is an educational architecture, not a description of any proprietary product.

1. Define what is being validated

Before designing a validation pipeline, distinguish five objects:

• Input: a value extracted from a document, spreadsheet, or user instruction.
• Assumption: the interpretation of that input used by the model.
• Evidence: the source that supports, qualifies, or contradicts the assumption.
• Model output: a result produced by formulas under specified assumptions.
• Professional decision: the judgment made after considering the output, uncertainty, and context.

These objects should not collapse into a single generated answer. Consider a hypothetical development feasibility model. An AI system extracts a construction cost from a project document and proposes entering it into a workbook.

The extraction may be correct, but that does not establish that the figure is current, applies to the relevant asset, or uses the correct unit. Even after the assumption has been approved, the resulting calculation still needs validation.

A useful validation system therefore asks separate questions:

1. Was the source read correctly?
2. Was the value interpreted correctly?
3. Is the assumption suitable for this model?
4. Did the calculation execute correctly?
5. Does the output behave as expected when inputs change?
6. Is the result sufficiently supported for professional review?

A single confidence score cannot adequately answer all six.

2. Keep language interpretation separate from calculation

Language models are useful for interpreting ambiguous instructions, extracting information, identifying relationships, and proposing structured data.

They should not be the sole authority for consequential arithmetic.

A safer design separates the workflow into distinct stages:

Source documents and workbook
            |
            v
AI-assisted extraction
            |
            v
Structured candidate assumptions
            |
            v
Schema and provenance validation
            |
            v
Human approval of material inputs
            |
            v
Deterministic calculation engine
            |
            v
Independent validation checks
            |
            v
Human review and release

This separation creates meaningful test boundaries.

For example, if a residual land value changes unexpectedly, an engineer can investigate whether the cause was an extracted input, a changed assumption, a formula defect, or a validation failure. The architecture does not guarantee that every error will be caught, but it makes errors easier to locate and investigate.

The calculation engine should receive typed, validated inputs and execute explicitly defined formulas. It should not need to infer the intended mathematics from a natural-language explanation each time a calculation runs.

3. Make assumptions machine-readable

A common weakness in AI-assisted workflows is treating generated prose as if it were a reliable data interface.

Instead, require the model to propose structured records. The exact schema will depend on the application, but a candidate assumption might look like this:

{
  "name": "construction_cost",
  "value": 1250,
  "unit": "currency_per_square_metre",
  "source_reference": "document_03",
  "source_location": "page_12",
  "status": "pending_review"
}

The number and unit above are illustrative, not market evidence.

A production schema would normally need additional fields, such as currency, applicable area definition, source date, geographic scope, extraction status, and approval metadata where relevant.

The important point is that the value is not accepted merely because the model returned valid JSON.

Validation should cover several layers:

Structural validation

• Are required fields present?
• Are types correct?
• Are unexpected fields rejected or explicitly handled?
• Are enumerated values valid?

Semantic validation

• Is the unit appropriate for the assumption?
• Is the value within a defensible range?
• Does the value match the intended asset and geography?
• Are required dates and scope definitions present?

Provenance validation

• Does the referenced source exist?
• Can the reviewer locate the supporting passage or cell?
• Does the source actually support the proposed interpretation?
• Is the evidence current enough for the intended use?

A schema can enforce structure. It cannot, by itself, establish that an assumption is commercially reasonable.

4. Validate calculations independently

Once assumptions are approved, calculations should run through deterministic code or a controlled spreadsheet calculation engine.

Deterministic means that the same defined inputs, formulas, and calculation rules produce the same result under the same execution conditions. It does not mean the model is economically correct.

Validation should include at least four categories.

Formula and dependency checks

Confirm that formulas use the intended cells, units, and dependency relationships. Detect broken references, unexpected hard-coded values, circular dependencies where prohibited, and missing required formulas.

Where a spreadsheet is involved, the validation layer should distinguish a formula cell from a displayed value. A cached result alone may not establish that the formula was recalculated using the latest inputs.

Accounting and reconciliation checks

Where applicable, test identities and reconciliations defined by the model specification.

Examples include:

• Total costs reconcile to their component categories.
• Financing sources and uses reconcile.
• Cash flows reconcile to the specified timing and period conventions.
• Intermediate outputs reconcile to downstream calculations.

The checks must be derived from the actual model specification. Not every financial model has the same identities or conventions.

Boundary and type checks

Reject or flag invalid inputs, incompatible units, missing required values, impossible dates, and calculations that divide by zero.

A value being numeric does not mean it is valid for the model.

Regression tests

Maintain a set of known test cases. After changing formulas, extraction logic, or calculation code, compare the results with approved expected outcomes.

Tests should cover normal cases, boundary conditions, missing data, and known failure scenarios. Expected results must come from independently checked calculations or approved specifications, not from the same model output being tested.

5. Test behavior, not just individual numbers

A model can pass isolated formula checks while behaving badly when assumptions change.

Sensitivity testing examines whether outputs respond in ways consistent with the model's defined relationships.

For a hypothetical development model, useful tests might include:

• Increasing a cost assumption while holding other inputs constant.
• Delaying revenue-generating activities.
• Reducing expected revenue.
• Changing financing assumptions within permitted ranges.
• Combining adverse assumptions to examine downside exposure.

These are diagnostic tests, not universal assertions about every model.

For example, increasing construction cost would generally be expected to reduce residual land value in a conventional residual valuation model when all other inputs remain fixed and the additional cost is not offset elsewhere. If the output increases instead, the system should flag the behavior for investigation.

The test does not automatically prove that the model is defective. There may be a legitimate dependency or offsetting rule. The purpose is to make the behavior explicit and explainable.

A practical test suite should distinguish:

• Invariant checks: conditions that must always hold.
• Expected directional checks: relationships that should hold under specified conditions.
• Scenario tests: outputs under deliberately changed assumptions.
• Regression tests: comparisons with previously verified cases.

Directional checks need careful design. Hard-coding intuitive rules without accounting for model dependencies can produce false alarms.

6. Treat human review as a defined control

Adding a human approval button is not sufficient. The reviewer needs the information and authority required to make a meaningful decision.

A useful review interface should expose:

• The proposed assumption and its unit.
• The original source and relevant location.
• The extraction or interpretation proposed by the AI.
• Any transformations applied to the source value.
• The downstream calculations affected.
• Validation failures and unresolved warnings.
• The difference between the current and proposed model state.

Review priority should reflect materiality and uncertainty. A minor formatting issue does not require the same treatment as an unsupported assumption that materially changes a financial result.

A possible approval policy is:

Passing automated checks should not be treated as proof that a model is suitable for a real-world decision.

The approval policy must also define who can approve an assumption, what evidence is required, how overrides are recorded, and when an approval expires or must be revisited.

7. Preserve an audit trail

A useful audit trail records the sequence of changes, not just the final workbook.

For each material assumption, record the source reference, extracted value, proposed interpretation, approved value, reviewer, timestamp, and reason for any override.

For each calculation run, record the model version, input version, calculation version, validation results, and output version.

Where AI is involved, also consider recording the model or service version and the relevant processing configuration, subject to privacy, security, and retention requirements.

This is important because an identical output can arise from different assumptions, while a changed output can result from either changed inputs or changed calculation logic.

Auditability requires enough information to reconstruct why a result was produced. A generated explanation that merely sounds plausible is not a substitute for that record.

8. Avoid common validation mistakes

Mistake 1: Treating valid JSON as correct data.

A schema proves that a response meets structural constraints. It does not prove that a source was interpreted correctly or that the value is appropriate.

Mistake 2: Asking the same AI system to verify its own answer.

A second model-generated explanation may identify useful issues, but it is not independent evidence of correctness. Independent checks should use source evidence, deterministic calculations, explicit invariants, or separately established expected results.

Mistake 3: Checking only the final output.

A plausible headline result can hide errors in intermediate calculations, units, timing, or assumptions. Validate the calculation chain as well as the final number.

Mistake 4: Treating human approval as a rubber stamp.

Reviewers need access to evidence and the ability to reject, amend, or escalate proposed assumptions. Otherwise, the control exists mainly on paper.

Mistake 5: Confusing model correctness with decision quality.

A model may calculate exactly as specified and still be unsuitable because its assumptions are weak, its evidence is stale, or relevant risks are omitted.

9. A practical implementation sequence

Teams building this capability can start with a narrow, testable scope.

1. Select one model and document its calculation rules.
2. Define a schema for a small set of material assumptions.
3. Require source references for those assumptions.
4. Validate types, units, required fields, and source availability.
5. Route material assumptions for human approval.
6. Run calculations outside the language-model interpretation step.
7. Implement reconciliation, boundary, regression, and sensitivity checks.
8. Record model versions, approvals, overrides, and validation outcomes.
9. Test the workflow against deliberately introduced errors.
10. Expand only after the failure modes and review burden are understood.

Evaluation should measure more than whether the pipeline completes. Useful measures include unsupported-assumption detection, source-traceability coverage, validation failures caught, false-positive rates, reviewer override frequency, and reproducibility across controlled test cases.

These measures need explicit definitions and representative test data. No performance figure should be claimed without a documented evaluation method.

Conclusion

Human-in-the-loop validation is most effective when it is designed into the system rather than added at the end.

AI can help interpret documents, propose structured assumptions, and explain relationships. Deterministic calculation, independent validation, source traceability, and accountable human approval address different parts of the problem.

The engineering objective is not to eliminate professional judgment. It is to ensure that the judgment is applied to a result whose assumptions, evidence, calculations, and limitations can be inspected.

For a financial model, a plausible number is only the beginning of the validation process.