跳到正文
DEV Community· Georgia Kali·· 3 小時前AI 評分55

IntDEx 如何在 AI SDLC 中按證據分級驗證行為

AI SDLC - Behaviour Validation

AI 導讀

IntDEx 框架將 AI SDLC 第 4 階段的行為驗證設為按證據來源分級的流程，並把模型自行聲稱測試通過的結果標記為未驗證。執行證據的順位高於人工證據、跨模型證據及先前工件比較；配置改動時，自動化持續行為驗證（ACBV）會重跑行為評估，並按通過率設置閘門。框架目前的應用涵蓋單一項目和單一主要評估者，作者將其定位為初步可行性證據，並指出仍需獨立多案例複現。

正文

Georgia Kali · 17 October 2026 · AI SDLC series, IntDEx framework

Why a self-reported "all tests pass" is not evidence, and how the IntDEx framework turns testing into evidence-ranked behaviour validation

The problem: when one model writes the code and the verdict

In an AI-assisted software development lifecycle (AI SDLC), the same model can now draft the intent, the prompt, the implementation, the tests and the final test report. The IntDEx framework v5 names the risk directly: a misunderstanding can be reproduced consistently through every one of those artefacts, so "traceability then appears complete, coverage full, and validation appears good, while the entire chain is wrong". Traceability between artefacts written by one author may be internal consistency, not verification.

This is not only a theoretical worry. NIST's Generative AI Profile (NIST AI 600-1) describes confabulation as confidently stated but erroneous or false content, and research presented at ICLR 2024 found that large language models struggle to correct their own reasoning without external feedback, with performance sometimes degrading after self-correction (Huang et al., 2024). A test phase that accepts the authoring model's own verdict inherits both weaknesses.

What IntDEx changes in Phase 4: Behaviour Validation

IntDEx (AI-native Delivery Experience) defines a seven-phase AI-native delivery flow. Phase 4, Behaviour Validation, "ensures that delivery engine's outputs are correct, safe, ethical, and aligned with stakeholder expectations", and user testing focuses on behaviour rather than code. Testers evaluate outputs for correctness, bias, hallucinations, constraint violations and drift.

Compared with conventional QA, UAT, test automation and evals, the framework makes four concrete changes:

  1. Evidential independence. A claim of correctness is admissible only if its evidence comes from a source other than the model that produced the work. Sources are ranked: executed evidence (exit codes, test results, build logs, HTTP responses, database state) above human evidence, above cross-model evidence, above prior-artefact comparison.
  2. Self-reports are UNVERIFIED. A statement by the authoring model that a check passed "carries no evidential weight and must be recorded as unverified rather than as a pass". The engine's own report is not evidence; the human decides.
  3. Pair behaviour validation. Acceptance validation is done in pairs, for example end user with tester, or user with analyst; security, UI/UX and compliance tests are automated before release and checked by human testers.
  4. Continuous validation. Automated Continuous Behaviour Validation (ACBV) re-runs behavioural evaluations whenever the engine configuration changes, whether a prompt, constraint, context or model swap, and gates the change on pass rates, so behavioural quality cannot degrade silently.

Research-style validation

Problem. Can behaviour validation in an AI SDLC produce assurance when the authoring model also writes the tests and the report?

Method. IntDEx was developed using Design Science Research, treating the framework as an artefact refined through practical application on a data-science platform for public-health data experimentation. Evaluation covered rebuilding the solution from artefacts alone and the operation of the evidential-independence and human-oversight controls.

How validation was applied. The v5 document describes applying these controls during the delivery of that platform and in an attempt to rebuild it from IntDEx artefacts alone. Enforced human checkpoints were used to halt work where the engine's own evidence was judged insufficient and to correct the severity the engine had assigned to a change. A model from a different provider family reviewed prompt executions, and its disagreements were passed to a human, who decided whether a change was needed. The rebuild was assessed across behaviour, security, data structures, architecture and configuration, and the authors note that security controls left implicit in the original, rather than written as governed intent or security artefacts, may not be reliably reproduced. Quantitative results will be reported in a forthcoming paper.

Worked example. Take the playbook's sample request: a website contact form that emails the business owner. Because it handles personal data, IntDEx treats it as a Tier 2 (elevated) change with a synchronous, blocking checkpoint. A validation record could look like this:

Acceptance criterion Evidence Rank Status
Valid enquiry is accepted and the owner is notified Integration test exit code and captured outbound message Executed Admissible
Abusive repeated submissions are throttled Engine states "rate limiting implemented" Self-report UNVERIFIED
Same criterion, after a negative test that floods the endpoint HTTP 429 responses logged Executed Admissible
Form wording meets the intent Recorded user and tester pair review Human Admissible

Proposed evaluation criteria (not results). Teams adopting this practice could track measures from the framework's own list: the percentage of production claims backed by independent evidence, behavioural regression, first-pass acceptance, escaped defects, and the negative-control-test pass rate. Because evaluations of a non-deterministic system vary between runs, IntDEx advises establishing the expected run-to-run variation and gating only on changes that exceed it.

Limitations. The authors state that the application so far covers a single project with a single principal evaluator. They present it as preliminary evidence of feasibility rather than general proof, and independent multi-case replication is still needed. Cross-model review is advisory, and in the authors' brief a second model helped but also raised false alarms. Human reviewers are a failure mode too. IntDEx names automation bias, a risk the EU AI Act's Article 14 also requires high-risk systems to address, and it treats a checkpoint rejection rate of zero as a finding that needs investigation, not as a sign of quality.

Conclusion. In IntDEx, behaviour validation is not "run the tests and trust the summary". It is an evidence ledger in which every claim names its source, self-assessment is labelled UNVERIFIED, and a named human accepts or rejects the evidence. For teams moving to an AI SDLC, that is a practical first step: rank the evidence before you trust the result.

References

IntDEx Framework

Other references

AI was used for readability improvements.

來源：DEV Community · dev.to

#教學/實踐#部署/工程