AI 生成財務模型的人工參與驗證方法

AI can generate a financial model that looks coherent, contains valid formulas, and produces a plausible result while still being wrong in ways that matter.

A formula can reference the wrong cell. A cost assumption can be expressed in the wrong unit. A revenue schedule can omit a phase. A model can calculate correctly from an unsupported market assumption.

These are different failure modes, and they need different controls. For real estate feasibility models, the objective should not be to ask whether an AI-generated workbook looks correct. It should be to establish which properties can be tested automatically, which assumptions require evidence, and which decisions must remain with a qualified human reviewer.

A useful validation architecture has five layers:

1. Structural validation
2. Deterministic calculation checks
3. Domain and consistency checks
4. Evidence and provenance checks
5. Human review and approval

No single layer is sufficient.

1. Separate the output from the evidence

Consider a hypothetical development model with the following inputs:

• Site area and developable area
• Permitted gross floor area
• Unit mix or lettable area
• Revenue assumptions
• Construction and other development costs
• Financing assumptions
• Development programme
• Exit assumptions

The model calculates development costs, revenue, cash flows, residual land value (RLV), and potentially internal rate of return (IRR).

For example, a model might calculate an RLV of £4 million. That output can be mathematically consistent with the workbook and still be commercially misleading if the revenue assumptions are unsupported or the construction cost basis is outdated. Validation therefore needs to test both the calculation and the justification for its inputs.

2. Layer one: validate the structure before the numbers

Before reviewing financial logic, validate the shape of the generated output. If an AI system is expected to return structured data before creating a workbook, define an explicit contract. JSON Schema is one established option for specifying required fields, data types, and constraints.

See the JSON Schema getting-started guide.
A simplified model input contract might look like this:
{
"project_id": "DEMO-001",
"currency": "GBP",
"site_area_sqm": 5000,
"gross_floor_area_sqm": 12000,
"construction_cost_per_sqm": 1800
}
This is illustrative data, not a market benchmark or a recommendation for a production schema.

A structural validator could check that:

• Required fields are present.
• Numeric values are actually numeric.
• Areas and costs use declared units.
• Currency is explicitly identified.
• Values fall within defined technical bounds.
• Unexpected fields are rejected or handled deliberately.
• Missing values are not silently replaced with zero.

The last point is important. Zero and unknown are not equivalent. If a finance rate is missing, substituting zero can make a model appear more profitable. A missing value should normally trigger a validation error or an explicit unresolved status.

Structural validation establishes that the model can be interpreted. It does not establish that the model is economically sensible.

3. Layer two: use deterministic checks for deterministic rules

Rules that can be expressed precisely should not depend on an LLM's judgment. For example, if a model defines construction cost as gross floor area multiplied by construction cost per square metre, the expected calculation is:

\text{GFA}
\times
\text{Cost per sqm}
]
For the hypothetical inputs above:
[
12{,}000 \times £1{,}800 = £21{,}600{,}000
]
A deterministic test can independently calculate the expected result and compare it with the workbook output.
from decimal import Decimal

gfa_sqm = Decimal("12000")
cost_per_sqm = Decimal("1800")
reported_cost = Decimal("21600000")

expected_cost = gfa_sqm * cost_per_sqm

assert expected_cost == reported_cost

This test checks arithmetic for the stated formula. It does not establish that the GFA is permitted, that the cost rate is appropriate, or that the formula captures every relevant cost.

A production implementation should also define tolerances where appropriate, particularly when comparing rounded spreadsheet values or calculations involving floating-point numbers.

Other deterministic checks might include:
Revenue equals the sum of its components.

• Total costs reconcile to their component schedules.
• Cash flow totals reconcile across periods.
• Unit counts and areas reconcile between schedules.
• The reported output matches an independently calculated result.
• A sensitivity case actually changes the intended input.
• Required formula cells are not unexpectedly blank or replaced by constants.

Where possible, use independent calculations rather than simply checking that a formula cell contains a formula. A formula can exist and still be wrong.

4. Layer three: validate model relationships

Many serious errors are not syntax errors or arithmetic errors. They are inconsistencies between parts of the model.

Suppose a development model contains:

• 120 residential units in the unit schedule
• 116 units in the sales schedule
• Revenue calculated using the sales schedule
• Construction costs calculated using the unit schedule

Every formula may evaluate successfully, but the workbook contains a discrepancy that needs resolution.

Domain-level checks can flag conditions such as:

• Unit counts that differ across schedules.
• Areas that exceed a declared model constraint.
• Revenue periods that do not align with the programme.
• Cost totals that exclude a populated cost category.
• An exit value that uses an inconsistent timing assumption.
• A sensitivity scenario that changes revenue but leaves dependent calculations unchanged.
• An IRR calculation using a cash-flow range that omits a period.

These checks should be encoded as explicit rules wherever the intended relationship is known.

Some rules require context. For example, whether a particular cost should be included in a development budget depends on the model's scope and conventions. A validator should not impose a universal rule where none exists.

Instead, it should identify the relevant condition, record the rule being applied, and send ambiguous cases for review.

5. Layer four: validate the provenance of assumptions

A financial model can pass every arithmetic test and still fail as an analytical tool.

The reason is straightforward: calculations cannot establish the truth of their inputs.

For every material assumption, a validation system should try to preserve:

• The value and unit.
• The source or origin of the value.
• The source's date and relevant data period.
• The geography and asset type, where applicable.
• The transformation from source evidence to model assumption.
• Any manual adjustment made by the analyst.
• The model locations that depend on the assumption.

Consider a construction cost assumption of £1,800 per square metre. A validator might confirm that the figure is numeric, the unit is recorded, and the workbook uses it consistently. Those checks do not establish whether the figure is suitable for the location, asset type, specification, and cost period. That question requires appropriate evidence and professional interpretation.

An LLM may help extract candidate assumptions from documents or explain how a value is used in a workbook. The extracted result should still be checked against its source, and the resulting assumption should remain distinguishable from the underlying evidence.
A useful principle is:
Traceability makes an assumption inspectable. It does not make the assumption correct.

6. Layer five: make human review a defined state transition

Human review should not be an informal final glance after an automated process has already treated the model as complete.

A practical workflow can represent validation explicitly:

GENERATED
|
v
STRUCTURE_CHECKED
|
v
CALCULATIONS_CHECKED
|
v
DOMAIN_CHECKED
|
v
EVIDENCE_REVIEWED
|
v
HUMAN_APPROVAL_REQUIRED
|
+----> APPROVED FOR THE DEFINED PURPOSE
|
+----> RETURNED FOR CORRECTION
|
+----> BLOCKED: UNRESOLVED ISSUE

The states are illustrative, but the distinction matters. A successful automated test should not automatically mean that a model is approved. Approval should be scoped to a defined version of the workbook, its assumptions, its evidence, and its intended use.
A reviewer should be able to see:

1. What was generated.
2. Which automated checks passed.
3. Which checks failed or were skipped.
4. Which assumptions lack adequate evidence.
5. What changed after a correction.
6. Who approved the model and for what purpose.

If a material input or formula changes after approval, the system should identify which checks need to run again and whether the earlier approval remains valid. This is where versioning and change tracking become part of model governance rather than administrative overhead.

7. Design the exception path before the success path

Validation is often designed around a clean example: the model is generated, every check passes, and the user receives the output. Production systems need to handle less convenient outcomes.

A failed check should produce actionable information. “Validation failed” is less useful than “The unit count in the sales schedule differs from the unit schedule; review the linked inputs before approving revenue.”

Similarly, a warning should not disappear simply because a later stage of the workflow succeeds. For auditability, preserve the check identifier, the model version, the result, the relevant evidence, and the resolution.

8. Measure validation quality without inventing an accuracy score

A single “model accuracy” percentage can hide important distinctions. A workbook might pass every arithmetic test while containing several unsupported assumptions. Another may contain a harmless formatting issue but have fully reconciled calculations and well-documented inputs.

A more useful validation report separates categories:

• Structural checks passed or failed.
• Deterministic calculation checks passed or failed.
• Domain checks passed, failed, or were not applicable.
• Assumptions with verified, missing, or conflicting evidence.
• Open exceptions and their severity.
• Human review status.
• The scope and version of the approved output.

If the system is evaluated over time, track false negatives and false positives for individual validation rules against a reviewed test set. Record the test cases, expected outcomes, rule versions, and known limitations.

Do not interpret a high pass rate as proof that the model is correct. It may simply mean that the checks cover only a narrow set of failure modes.

The purpose of validation is to make known risks visible and testable, not to manufacture certainty.

9. A practical implementation sequence

A team introducing validation around AI-generated financial models does not need to automate every review step at once.

A sensible sequence is:

1. Define the contract. Specify the required inputs, output fields, units, and missing-value behaviour.
2. Write deterministic tests. Start with totals, reconciliations, formula ranges, and independent calculations.
3. Add domain checks. Encode known cross-schedule relationships and clearly defined constraints.
4. Track provenance. Associate material assumptions with sources, dates, units, and transformations.
5. Introduce exception handling. Make failures explicit and prevent unresolved material issues from being silently treated as passed.
6. Define human approval. Record the reviewer, workbook version, purpose, and outstanding qualifications.
7. Build a regression suite. Retain representative valid and invalid examples and rerun the checks when the generation logic or validation rules change.

The test suite should include deliberate failures, not only successful examples. Examples might include missing finance assumptions, inconsistent unit counts, a formula referencing the wrong period, and a source whose geography does not match the project.

This helps test whether the validation process detects the intended problem rather than merely accepting a plausible-looking output.

Conclusion

Human-in-the-loop validation is not a person checking an AI result after the important work is finished. It is a system design in which deterministic rules handle deterministic questions, domain checks expose inconsistencies, evidence is kept separate from assumptions, and a human reviewer remains accountable for unresolved judgments.

For financial modelling, that distinction is material. A workbook can be well-formed without being economically sound, mathematically consistent without being supported by market evidence, and ready for review without being ready for a decision.

The objective is not to make every AI-generated model look trustworthy. It is to make the model's logic, assumptions, limitations, and unresolved issues sufficiently visible that a professional can challenge it before relying on the result.