工程領袖如何為 LLM 工作流設定可靠性目標
How Engineering Leaders Should Set Reliability Targets for LLM Workflows
工程領袖應以業務結果而非單看基礎設施可用性,為 LLM 工作流設定可靠性目標,並在完整工作流層面定義 SLI、SLO 和錯誤預算。衡量範圍涵蓋任務成功、服務可用性、回應品質與依據、執行正確性、延遲、安全、復原能力及營運成本。文中以 Python 範例展示如何計算任務成功率、延遲合規率及錯誤預算消耗,並指出安全違規須獨立處理,不應納入一般容錯預算。
A practical engineering guide to defining service-level objectives, error budgets, and measurable reliability requirements for production AI applications.
Engineering leaders should define reliability targets for LLM workflows around successful business outcomes rather than infrastructure availability alone.
A model endpoint may return successful HTTP responses while generating incorrect answers, retrieving irrelevant documents, selecting inappropriate tools, or failing to complete the intended business operation.
Traditional reliability indicators such as uptime, latency, and request error rates remain important. However, LLM applications also require measurements covering task correctness, grounding, authorization, execution reliability, recovery, and operating costs.
The most practical approach is to define service-level indicators for the complete workflow, establish measurable service-level objectives, and use error budgets to guide deployment and engineering decisions.
These targets should reflect the consequences of failure, user expectations, operational dependencies, and the organization's ability to detect and recover from problems.
An AI workflow is reliable when it consistently produces an acceptable outcome under its intended operating conditions.
1. Why Conventional Uptime Targets Are Insufficient
Traditional application reliability often begins with questions such as:
- Is the service available?
- Are requests completing successfully?
- Is response latency within acceptable limits?
- Are infrastructure dependencies operational?
These questions remain relevant to AI applications.
However, they do not establish whether the application is performing its intended task correctly.
Consider an enterprise knowledge assistant.
A request completes successfully in two seconds and returns an HTTP 200 response.
From an infrastructure perspective, the request appears successful.
But the assistant may have:
- Generated an unsupported answer.
- Retrieved outdated documentation.
- Misinterpreted the user's question.
- Used information from an unauthorized source.
- Failed to identify conflicting evidence.
The infrastructure is available, but the business outcome is unreliable.
Now consider an AI agent responsible for creating support tickets.
The agent returns a successful response confirming that a ticket has been created.
However, an integration timeout caused the agent to retry the operation, creating two identical tickets.
Again, the request appears successful while the underlying business operation has failed.
Engineering leaders must therefore distinguish between technical availability and workflow reliability.
2. Define Reliability at the Workflow Level
LLM applications rarely consist of a single model call.
A production workflow may include:
- Request authentication.
- Input validation.
- Intent classification.
- Document retrieval.
- Context assembly.
- Model inference.
- Tool selection.
- External API execution.
- Output validation.
- Response delivery.
Each component can fail independently.
A workflow can also fail because of interactions between components that individually appear healthy.
For example, a retrieval service may return documents successfully, but those documents may not contain the information required to answer the question.
The model then produces a plausible but unsupported response.
Neither component necessarily reports a conventional infrastructure error.
Component reliability
Component reliability measures the behavior of an individual service.
Examples include:
- Model provider availability.
- Vector database latency.
- Retrieval service error rate.
- Tool execution success.
- Output validation failures.
Workflow reliability
Workflow reliability measures whether the complete process achieves an acceptable outcome.
Examples include:
- A support request is correctly classified and routed.
- A document question receives a grounded answer.
- An invoice is extracted and validated correctly.
- An agent completes an authorized business operation without duplication.
Both measurement levels are necessary.
Component indicators help engineers locate failures.
Workflow indicators establish whether those failures affect users or business operations.
3. Understand SLIs, SLOs, and SLAs for LLM Applications
Site Reliability Engineering provides three useful concepts for defining reliability expectations.
Service-Level Indicator (SLI)
An SLI is a quantitative measurement of service behavior.
For an LLM workflow, examples include:
- Percentage of eligible tasks completed successfully.
- Percentage of requests completed within a latency threshold.
- Percentage of evaluated answers supported by authoritative evidence.
- Percentage of tool operations executed correctly.
An SLI measures what actually happened.
Service-Level Objective (SLO)
An SLO defines the desired target for an SLI over a specified measurement window.
For example:
At least 99% of eligible document-processing requests must complete successfully over a rolling 30-day period.
The target is illustrative. Its suitability depends on the workflow.
Service-Level Agreement (SLA)
An SLA represents a service commitment that may have contractual or other defined consequences if the agreed requirements are not met.
An organization may establish internal SLOs that are stricter than its external SLA commitments.
AI application teams should avoid making contractual quality guarantees that they cannot reliably measure or enforce.
Google's Site Reliability Engineering guidance explains how service-level objectives can be used to align reliability engineering with user expectations and business requirements.
Reference: https://sre.google/sre-book/service-level-objectives/
4. Define What Counts as a Successful AI Workflow
Before selecting reliability targets, engineering teams need an explicit definition of success.
This is more difficult for AI applications than for conventional API endpoints.
A generated response may be grammatically correct but factually inaccurate.
A tool call may succeed technically but perform the wrong operation.
A retrieval request may return relevant information while violating document permissions.
Example: Enterprise RAG assistant
A successful task might require:
- The request is authorized.
- The appropriate documents are retrieved.
- The answer addresses the user's question.
- Material factual claims are supported by retrieved evidence.
- Information disclosure policies are satisfied.
- The response completes within an acceptable time.
If available evidence is insufficient, a correct abstention or escalation may also represent an acceptable outcome.
The definition should reflect what the application is designed to accomplish.
Example: AI-powered invoice processing
A successful task might require:
- Required fields are extracted correctly.
- Vendor and purchase-order information is validated.
- Business rules are applied.
- Duplicate processing is prevented.
- Exceptions are routed appropriately.
- Required approvals are enforced.
A workflow that correctly sends an invalid invoice for human review may count as successful.
Automatically approving an invalid invoice should not.
Example: AI support agent
A successful task might require:
- The user's intent is identified.
- Appropriate information is retrieved.
- Authorized tools are selected.
- Tool arguments are valid.
- The requested operation is completed correctly.
- No unauthorized side effects occur.
- The final response accurately reflects the resulting business state.
Engineering leaders should document success definitions before setting numerical targets.
Otherwise, teams may optimize measurements that do not represent actual reliability.
5. Eight Reliability Dimensions for Production LLM Workflows
The following eight dimensions provide a practical starting point for establishing reliability objectives.
Dimension 1: End-to-End Task Success
Question: Does the workflow achieve its intended outcome?
Task success should be the primary reliability indicator for many AI applications.
A general formula is:
Task Success Rate = Successfully Completed Eligible Tasks / Total Eligible Tasks × 100
Eligibility must be explicitly defined.
For example, malformed requests from unauthorized users may be excluded from a business task success metric, while valid requests that fail because of provider outages must remain included.
Avoid excluding failed tasks simply because they encountered operational problems.
Where task correctness cannot be automatically established, use representative evaluation samples and qualified human review.
Dimension 2: Service Availability
Question: Can eligible users access and use the workflow when required?
Availability measures whether the application can accept and process requests according to its service requirements.
Relevant failures include:
- Application outages.
- Model provider unavailability.
- Retrieval service failures.
- Authentication dependency failures.
- Infrastructure errors.
- Unrecoverable integration problems.
Availability should be defined from the user's perspective rather than relying exclusively on individual component health checks.
A healthy API gateway does not mean the AI workflow is available.
Dimension 3: Response Quality and Grounding
Question: Are generated outputs sufficiently correct and supported?
For knowledge-based applications, relevant indicators may include:
- Answer correctness.
- Grounded claim rate.
- Citation correctness.
- Unsupported claim frequency.
- Correct abstention behavior.
- Instruction-following performance.
Quality measurements should use clearly defined evaluation criteria.
LLM-based evaluators can assist with semantic assessment, but their results should be calibrated against human-reviewed examples.
For applications handling sensitive decisions, critical quality failures should receive separate attention rather than being absorbed into an aggregate score.
Dimension 4: Execution Correctness
Question: Do tool calls and automated workflows produce the intended business state?
This dimension is particularly important for AI agents.
Measure:
- Correct tool selection.
- Valid tool arguments.
- Successful authorized operations.
- Duplicate-action frequency.
- Correct execution ordering.
- Business-state consistency.
A successful API response is not sufficient evidence of execution correctness.
The evaluation should verify the resulting state in the relevant enterprise system.
For example, an agent creating a support ticket should verify that the correct ticket exists with the expected information and that duplicate records were not created.
Dimension 5: Latency and Responsiveness
Question: Does the workflow complete within acceptable time limits?
LLM workflow latency depends on several components.
These may include retrieval, reranking, inference, external tools, retries, and output validation.
Useful indicators include:
- End-to-end latency.
- Time to first token.
- Model inference duration.
- Tool execution duration.
- Queue waiting time.
- p95 and p99 latency.
The appropriate measurement depends on the user experience.
A conversational assistant may require responsive streaming.
An asynchronous document-processing workflow may tolerate longer completion times.
Targets should reflect the business process rather than an arbitrary technical benchmark.
Dimension 6: Security and Policy Enforcement
Question: Does the workflow remain within authorized operating boundaries?
Security indicators may include:
- Unauthorized retrieval events.
- Unapproved tool execution.
- Cross-tenant information exposure.
- Approval bypass events.
- Prohibited data transfers.
- Critical policy violations.
Mandatory security requirements should not be treated as ordinary tolerable errors.
For example, a 99% task success target does not justify unauthorized access in the remaining 1% of requests.
An error budget for availability or answer quality must not become a budget for security violations.
OWASP's generative AI security guidance provides relevant threat categories for defining security requirements and evaluation scenarios.
Reference: https://genai.owasp.org/llm-top-10/
Dimension 7: Recovery and Resilience
Question: Can the application recover appropriately when dependencies fail?
Relevant indicators include:
- Successful recovery from transient failures.
- Retry exhaustion frequency.
- Fallback success rate.
- Interrupted workflow recovery.
- Incident detection time.
- Service restoration time.
AI applications may depend on external model providers, retrieval services, and enterprise APIs.
Recovery behavior should be designed explicitly.
An agent that retries indefinitely is not reliable simply because it continues attempting the task.
Bounded retries, persistent state, idempotency, and safe fallback behavior are important foundations.
Dimension 8: Operational Cost Predictability
Question: Can the application maintain its reliability targets within approved operating costs?
Cost is not identical to reliability, but it can operate as an important engineering constraint.
An application may increase task success by making additional model calls or repeatedly invoking expensive evaluation services.
The resulting architecture may become economically unsustainable.
Track:
- Model calls per task.
- Token consumption.
- Infrastructure expenses.
- External API costs.
- Human review costs.
- Cost per successful task.
A useful formula is:
Cost per Successful Task = Total Relevant Operating Cost / Successfully Completed Tasks
Engineering leaders should evaluate reliability improvements alongside their financial implications.
6. Set Targets According to Workflow Risk and User Expectations
Not every LLM application requires the same reliability objectives.
An internal summarization tool has different operating requirements from an AI workflow that creates financial records or interacts with critical infrastructure.
Category A: Low-Impact Assistive Workflows
Examples include:
- Drafting internal content.
- Summarizing non-sensitive documentation.
- Generating brainstorming suggestions.
Typical priorities include:
- Response usefulness.
- Reasonable availability.
- Acceptable responsiveness.
- Predictable operating costs.
Users may tolerate some incorrect or incomplete outputs when review is straightforward and consequences are limited.
Category B: Business-Critical Knowledge Workflows
Examples include:
- Customer support knowledge assistants.
- Enterprise policy search.
- Technical troubleshooting assistants.
- Internal operational guidance.
Typical priorities include:
- Answer correctness.
- Grounding.
- Data freshness.
- Permission enforcement.
- Consistent response latency.
- Appropriate abstention.
Incorrect information may affect important business processes even when the application does not directly execute actions.
Category C: Transactional AI Workflows
Examples include:
- Invoice processing.
- CRM record updates.
- Order-management automation.
- Support ticket creation.
Typical priorities include:
- Correct execution.
- Authorization.
- Idempotency.
- Business-state validation.
- Reliable exception handling.
- Recovery from partial failures.
These workflows require stronger controls around side effects.
Category D: High-Impact or Regulated Workflows
Examples may include applications supporting financial decisions, healthcare operations, sensitive employment processes, or privileged infrastructure operations.
Typical priorities include:
- Strict access enforcement.
- Validated human oversight.
- Critical failure prevention.
- Detailed auditability.
- Strong operational controls.
- Domain-specific evaluation.
Additional legal, contractual, and regulatory requirements may apply.
Reliability targets alone do not establish that such applications are safe or compliant.
7. Create an Explicit Reliability Contract
A reliability contract documents how engineering teams measure and maintain the expected behavior of a workflow.
Each SLO should specify:
- The workflow or service being measured.
- The user population or eligible request scope.
- What constitutes a good event.
- What constitutes a bad event.
- The target.
- The measurement window.
- The measurement source.
- The accountable owner.
- The action taken when the target is breached.
Example reliability contract
Consider a hypothetical enterprise support assistant.
Workflow: Internal support knowledge retrieval.
Primary SLI: Percentage of eligible requests producing an acceptable answer or an appropriate escalation.
Illustrative SLO: 97% over a rolling 30-day period.
Latency objective: At least 95% of eligible requests complete within four seconds.
Quality objective: At least 98% of evaluated factual claims are supported by approved evidence.
Mandatory security requirement: No observed unauthorized document disclosures.
Owner: Enterprise AI application team.
Measurement source: Workflow events, validated evaluation samples, security tests, and incident records.
Breach response: Investigate error-budget consumption, restrict affected releases where appropriate, and prioritize remediation.
These targets are illustrative.
A production team should establish its own thresholds based on baseline evidence, user expectations, risk assessment, and operational feasibility.
The contract must also distinguish between metrics collected across all requests and quality measurements estimated from sampled evaluations.
A sampled grounding metric should not be presented as though every production response has been independently verified.
8. Calculate Error Budgets for LLM Workflows
An error budget represents the amount of failure permitted under a defined service-level objective.
It helps engineering leaders balance feature delivery with reliability investment.
Example: Request-based error budget
Assume an AI workflow has the following illustrative SLO:
99% of eligible tasks must complete successfully during a 30-day measurement window.
During that period, the application processes 100,000 eligible tasks.
The permitted failure proportion is 1%.
Therefore:
Error Budget = 100,000 × 0.01
Permitted unsuccessful tasks = 1,000
If the workflow experiences 350 unsuccessful tasks, it has consumed 35% of its error budget.
The remaining budget is 650 unsuccessful tasks, assuming the stated total request volume.
Why the measurement unit matters
A request-based error budget is different from a time-based availability budget.
For example, a 99.9% time-based availability objective over a 30-day period corresponds to approximately 43.2 minutes of permitted unavailability.
That calculation should not be used to estimate allowable incorrect answers or failed agent actions.
The budget must use the same measurement unit as the associated SLI.
Maintain separate budgets for different objectives
An enterprise AI application could maintain independent budgets for:
- Unsuccessful tasks.
- Requests exceeding latency limits.
- Technical availability failures.
- Measured quality regressions.
Combining every failure category into one metric may obscure important problems.
Critical security violations and other mandatory requirements should have separate incident and release-blocking policies rather than a conventional tolerable failure budget.
Google's SRE Workbook provides guidance on using error-budget policies to establish actions when reliability objectives are threatened.
Reference: https://sre.google/workbook/error-budget-policy/
9. Implement Reliability Measurement in Python
The following example demonstrates how to evaluate a request-based reliability objective using Python 3.11 or later.
It requires no external dependencies.
The example assumes that a trusted evaluation process has already determined whether each workflow event meets the applicable success criteria.
Step 1: Define workflow events
from dataclasses import dataclass
@dataclass(frozen=True)
class WorkflowEvent:
eligible: bool
task_success: bool
latency_ms: float
critical_violation: bool = False
Each event records whether the request is eligible, whether the task succeeded, its latency, and whether a critical violation was observed.
In production, these events would typically come from an application telemetry pipeline or evaluation service.
Step 2: Define the reliability policy
@dataclass(frozen=True)
class ReliabilityPolicy:
task_success_target: float = 0.99
latency_target: float = 0.95
latency_threshold_ms: float = 4000
This illustrative policy establishes two separate objectives:
- At least 99% successful eligible tasks.
- At least 95% of eligible tasks completed within four seconds.
Any observed critical violation will be handled separately.
Step 3: Define the reliability report
@dataclass(frozen=True)
class ReliabilityReport:
eligible_tasks: int
successful_tasks: int
unsuccessful_tasks: int
task_success_rate: float
latency_compliance_rate: float
permitted_failures: float
error_budget_consumed: float
critical_violations: int
objectives_met: bool
The report provides aggregate reliability measurements alongside information about mandatory violations.
Step 4: Calculate the reliability indicators
from math import isfinite
def evaluate_reliability(
events: list[WorkflowEvent],
policy: ReliabilityPolicy,
) -> ReliabilityReport:
if not 0 < policy.task_success_target < 1:
raise ValueError("Invalid task success target")
if not 0 < policy.latency_target <= 1:
raise ValueError("Invalid latency target")
if (
not isfinite(policy.latency_threshold_ms)
or policy.latency_threshold_ms <= 0
):
raise ValueError("Invalid latency threshold")
eligible_events = [
event
for event in events
if event.eligible
]
total = len(eligible_events)
if total == 0:
raise ValueError("No eligible workflow events")
for event in eligible_events:
if (
not isfinite(event.latency_ms)
or event.latency_ms < 0
):
raise ValueError("Invalid event latency")
successful = sum(
event.task_success
and not event.critical_violation
for event in eligible_events
)
unsuccessful = total - successful
within_latency = sum(
event.latency_ms <= policy.latency_threshold_ms
for event in eligible_events
)
critical_violations = sum(
event.critical_violation
for event in events
)
success_rate = successful / total
latency_rate = within_latency / total
permitted_failures = (
total * (1 - policy.task_success_target)
)
budget_consumed = (
unsuccessful / permitted_failures
)
objectives_met = (
success_rate >= policy.task_success_target
and latency_rate >= policy.latency_target
and critical_violations == 0
)
return ReliabilityReport(
eligible_tasks=total,
successful_tasks=successful,
unsuccessful_tasks=unsuccessful,
task_success_rate=success_rate,
latency_compliance_rate=latency_rate,
permitted_failures=permitted_failures,
error_budget_consumed=budget_consumed,
critical_violations=critical_violations,
objectives_met=objectives_met,
)
Critical violations are counted across all supplied events, including events excluded from the primary task-success population.
This prevents eligibility filtering from automatically concealing security incidents.
The permitted failure value is calculated continuously from the observed request volume. A real policy may additionally define integer rounding and minimum sample requirements.
Step 5: Evaluate a hypothetical workload
events = [
WorkflowEvent(
eligible=True,
task_success=True,
latency_ms=1800,
)
for _ in range(990)
]
events.extend([
WorkflowEvent(
eligible=True,
task_success=False,
latency_ms=2500,
)
for _ in range(10)
])
report = evaluate_reliability(
events,
ReliabilityPolicy(),
)
print(
f"Task success: "
f"{report.task_success_rate:.2%}"
)
print(
f"Latency compliance: "
f"{report.latency_compliance_rate:.2%}"
)
print(
f"Error budget consumed: "
f"{report.error_budget_consumed:.2%}"
)
print(
f"Objectives met: "
f"{report.objectives_met}"
)
Expected output:
Task success: 99.00%
Latency compliance: 100.00%
Error budget consumed: 100.00%
Objectives met: True
The workflow meets its stated objectives but has consumed its complete task-success error budget for the evaluated event set.
This distinction is important.
Meeting an SLO at its minimum boundary does not necessarily mean the service is in a healthy position for additional risky changes.
Production limitations
This example does not implement:
- Rolling measurement windows.
- Persistent telemetry storage.
- Delayed quality labels.
- Event deduplication.
- Statistical confidence intervals.
- Distributed tracing.
- Error-budget burn-rate alerts.
- Monitoring of missing telemetry.
- Separate sampling-based quality objectives.
A production implementation must address these requirements where applicable.
The example demonstrates the relationship between workflow events, reliability objectives, and error-budget consumption.
10. Use Error-Budget Burn Rates to Detect Emerging Problems
An error budget tells engineering teams how much failure is permitted.
A burn rate indicates how quickly the service is consuming that budget.
The basic relationship is:
Burn Rate = Observed Failure Rate / Permitted Failure Rate
Consider a workflow with a 99% successful-task objective.
Its permitted failure rate is 1%.
If 5% of tasks fail during a recent observation window:
Burn Rate = 5% / 1% = 5
The workflow is consuming its request-based error budget at five times the sustainable failure proportion.
Why multiple observation windows matter
A brief failure spike may resolve quickly.
A lower but persistent failure rate may consume the entire budget over time.
Engineering teams can use short and longer observation windows to distinguish urgent incidents from sustained deterioration.
The exact alert thresholds should reflect request volume, measurement uncertainty, and operational requirements.
Example alerting situations
A short-window alert might detect:
- A model provider outage.
- A newly introduced tool execution failure.
- A broken retrieval integration.
- A deployment causing malformed outputs.
A longer-window alert might detect:
- Gradually declining answer quality.
- Increasing retrieval failures.
- Persistent latency degradation.
- Growing exception-handling problems.
Google's SRE guidance discusses multi-window, multi-burn-rate alerting for service-level objectives.
Reference: https://sre.google/workbook/alerting-on-slos/
For LLM quality indicators measured through delayed or sampled evaluation, immediate burn-rate alerting may not always be appropriate.
The alerting architecture should reflect how quickly reliable quality evidence becomes available.
11. Account for Multi-Step Workflow Dependencies
Production LLM workflows commonly depend on several services.
Consider a simplified application with four required stages:
- Authentication and request processing.
- Document retrieval.
- Model inference.
- Output validation.
Suppose each stage has an illustrative success probability of 99.9%.
If the failures are independent and every stage must succeed on a single attempt, the theoretical workflow success probability is:
0.999 × 0.999 × 0.999 × 0.999
= 0.996005996
≈ 99.60%
Even highly reliable individual components can produce a lower end-to-end success probability when several are required for every task.
Important limitations of this calculation
Real production failures are not necessarily independent.
Components may share infrastructure, network dependencies, identity providers, or model services.
Retries and fallbacks can also change effective success probabilities.
Therefore, multiplying individual component reliability figures is useful for illustrating dependency risk, but it should not replace actual workflow measurements.
Engineering considerations
When setting an end-to-end reliability objective, identify:
- Required synchronous dependencies.
- Optional dependencies.
- Shared infrastructure.
- Single points of failure.
- External provider limits.
- Sequential agent operations.
- Recovery paths.
- Alternative processing options.
A component's advertised availability does not directly establish the reliability of the complete AI workflow.
12. Design Fallbacks Around Acceptable Outcomes
A fallback should preserve an acceptable business outcome, not simply produce another response.
Consider a RAG assistant experiencing a retrieval service failure.
One possible fallback is to ask the LLM to answer without retrieved context.
However, this may violate the application's grounding requirements.
A more appropriate fallback could be:
- Return a transparent temporary-unavailability response.
- Offer approved static documentation.
- Route the question to a human support team.
- Request the user to retry later.
The correct fallback depends on the workflow's success definition.
Model-provider fallback
Switching to another model may improve technical availability.
However, the alternative model must meet relevant requirements for:
- Data-processing authorization.
- Output compatibility.
- Task quality.
- Tool execution.
- Security behavior.
- Operational costs.
An unapproved provider should not receive confidential enterprise information simply because the preferred provider is unavailable.
Agent execution fallback
If a tool operation fails, the agent may attempt recovery.
Before retrying, the execution service should determine whether the original operation was already completed.
For example, an external API timeout does not establish that the requested transaction failed.
Blindly repeating the operation may create duplicate records or financial side effects.
Use idempotency controls and state reconciliation where applicable.
Measure fallback outcomes separately
A useful monitoring strategy distinguishes between:
- Normal successful completion.
- Successful completion through fallback.
- Correct abstention.
- Appropriate escalation.
- Unsuccessful completion.
A rising fallback rate may reveal deteriorating primary-service reliability even when the overall workflow continues meeting its SLO.
Human escalation should count as a successful outcome only when it satisfies the agreed workflow requirements.
Otherwise, excessive escalation could conceal weak automation performance.
13. Connect Reliability Objectives to Release Decisions
Reliability targets should influence engineering decisions rather than remain dashboard metrics.
Before deploying changes to models, prompts, retrieval configurations, or tools, teams should evaluate whether the candidate release threatens existing objectives.
Example: Release eligibility policy
A release may require:
- Core workflow regression tests pass.
- No observed mandatory security violations.
- Candidate quality meets minimum acceptance requirements.
- Performance remains within the approved objective.
- Recovery and rollback procedures are available.
- Error-budget consumption is within the approved policy.
The final condition links current operational reliability with future release risk.
Implement a basic release policy
def release_blockers(
current: ReliabilityReport,
critical_regression_failures: int,
rollback_ready: bool,
emergency_reliability_fix: bool = False,
) -> list[str]:
blockers = []
if current.critical_violations > 0:
blockers.append(
"Unresolved critical violations require review"
)
if critical_regression_failures > 0:
blockers.append(
"Critical regression tests failed"
)
if not rollback_ready:
blockers.append(
"Rollback procedure is unavailable"
)
if not emergency_reliability_fix:
if current.error_budget_consumed >= 1:
blockers.append(
"Task-success error budget exhausted"
)
if not current.objectives_met:
blockers.append(
"Current reliability objectives are unmet"
)
return blockers
For the earlier workload, a normal feature release would be blocked because the task-success error budget has been fully consumed.
The emergency reliability fix parameter allows a simplified exception to the ordinary budget restriction.
It does not bypass critical regression requirements or unresolved critical violations.
Production considerations
A complete release policy should distinguish ordinary feature changes from security patches, incident remediation, and rollbacks.
Corrective deployments may be necessary precisely because the service is unreliable.
Emergency changes should follow a separate, controlled approval process rather than relying on an unprotected Boolean parameter.
The code also treats observed critical violations in the supplied report conservatively. A real gate should consult the current incident and remediation status instead of indefinitely blocking releases because of previously resolved historical events.
14. Establish an Observability Architecture for Reliability Measurement
Engineering leaders need reliable telemetry before they can enforce meaningful SLOs.
A practical architecture connects application traces, workflow outcomes, evaluation results, and operational monitoring.
User Request
|
v
Application Gateway
|
v
Workflow Orchestrator
|
+--> Retrieval Service
|
+--> Model Gateway
|
+--> Tool Execution
|
+--> Output Validation
|
v
Workflow Outcome Event
|
v
Evaluation and Telemetry Pipeline
|
+--> Task Success Metrics
|
+--> Quality Evaluation
|
+--> Latency Metrics
|
+--> Security Events
|
+--> Cost Metrics
|
v
SLO and Error-Budget Monitoring
|
v
Alerts and Engineering Decisions
The architecture should allow engineers to connect a failed business outcome to the relevant component traces.
OpenTelemetry provides semantic conventions for generative AI operations that can support consistent instrumentation.
Reference: https://opentelemetry.io/docs/specs/semconv/gen-ai/
Recommended workflow event attributes
Useful attributes may include:
- Workflow identifier.
- Request identifier.
- Application release version.
- Model deployment.
- Prompt version.
- Retrieval configuration.
- Tool execution status.
- Completion state.
- Latency.
- Retry count.
- Fallback classification.
- Evaluation status.
- Policy violation indicator.
Sensitive information should not be recorded indiscriminately.
Raw prompts, retrieved documents, tool arguments, and model responses require appropriate access controls and retention policies.
Monitor the measurement system itself
Reliability reports may become misleading if telemetry is incomplete.
Engineering teams should monitor:
- Missing workflow completion events.
- Delayed evaluation results.
- Duplicate telemetry records.
- Instrumentation failures.
- Changes in evaluation coverage.
- Unclassified workflow outcomes.
A service should not appear more reliable simply because its measurement system stopped recording failures.
15. Set Reliability Targets When Historical Data Is Limited
New AI applications often lack sufficient production history.
This makes it difficult to establish defensible long-term objectives.
A practical approach is to introduce reliability targets progressively.
Step 1: Define mandatory constraints
Identify requirements that apply regardless of historical performance.
Examples include authorization, data isolation, approval enforcement, and prohibited business operations.
Step 2: Establish an initial baseline
Run representative evaluation tasks and controlled workload tests.
Measure:
- Task success.
- Quality.
- Latency.
- Execution correctness.
- Recovery behavior.
- Operating costs.
Step 3: Identify business expectations
Ask stakeholders which failures would materially affect operations.
A support assistant and a financial processing application will have different tolerances.
Step 4: Establish provisional objectives
Define initial targets using the available evidence and approved risk constraints.
Document that these targets are provisional.
Step 5: Launch within a controlled scope
Restrict the initial deployment where appropriate.
Collect operational evidence and investigate unexpected failure patterns.
Step 6: Reassess the objectives
Update targets when sufficient evidence becomes available.
Changes should follow documented approval and review procedures.
An organization should not lower its reliability targets simply because the current implementation is struggling to meet them.
The appropriate response may instead involve architecture changes, reduced deployment scope, or additional engineering investment.
16. Connect Reliability Targets to Engineering Investment
SLOs help engineering leaders identify where reliability improvements create meaningful value.
For example, a workflow may experience several types of failure.
Retrieval-related failures
Potential investments include:
- Improving document ingestion.
- Correcting metadata.
- Updating retrieval strategies.
- Improving source freshness.
- Strengthening ranking evaluation.
Model-related failures
Potential investments include:
- Better prompt design.
- Model selection.
- Structured output validation.
- Context improvements.
- Appropriate model routing.
Tool execution failures
Potential investments include:
- Stronger argument validation.
- Idempotency controls.
- Transaction-safe execution.
- Improved retry policies.
- Better integration monitoring.
Infrastructure failures
Potential investments include:
- Capacity improvements.
- Queue management.
- Dependency isolation.
- Controlled failover.
- Deployment improvements.
Operational failures
Potential investments include:
- Improved incident detection.
- Clearer service ownership.
- Better runbooks.
- On-call readiness.
- Automated recovery mechanisms.
Engineering leaders should prioritize improvements according to their contribution to end-to-end reliability rather than optimizing whichever component has the most visible dashboard.
Where external engineering expertise can help
Organizations moving LLM applications from experimentation into production may need additional capabilities across AI engineering, software architecture, integration development, cloud infrastructure, and operational reliability.
CodingCops provides AI development and consulting, custom software engineering, dedicated engineering teams, and cloud and DevOps services.
These service areas are relevant when enterprises need to address reliability across the complete AI application rather than focus exclusively on model performance.
Regardless of whether the organization uses internal teams or external engineering partners, reliability objectives, risk acceptance, and operational accountability should remain explicitly defined.
17. Example: Setting Reliability Targets for an Enterprise Support Agent
Consider a hypothetical enterprise deploying an AI-powered support agent.
The agent can:
- Retrieve internal troubleshooting documentation.
- Answer employee questions.
- Create support tickets.
- Escalate complex issues.
- Check the status of existing requests.
The organization initially monitors API availability and average response latency.
These indicators suggest that the application is operating normally.
However, production reviews identify several problems.
The agent sometimes retrieves outdated troubleshooting instructions.
Some support tickets contain incomplete information.
Occasional integration timeouts create duplicate-action attempts.
Employees also report that escalation paths are inconsistent.
Revised reliability strategy
The engineering team introduces several distinct objectives.
Task success objective
At least 97% of eligible support requests should achieve an acceptable automated resolution or appropriate escalation under the defined service contract.
Grounding objective
At least 98% of sampled material factual claims should be supported by approved documentation.
Latency objective
At least 95% of interactive requests should complete within five seconds, excluding separately defined asynchronous operations.
Execution correctness objective
At least 99.5% of eligible ticket-creation operations should produce the intended valid ticket state.
Mandatory controls
No observed unauthorized ticket access, prohibited data disclosure, or unauthorized consequential actions.
These percentages are hypothetical and are not recommendations for every support deployment.
Engineering changes
To support the objectives, the team introduces:
- Document freshness monitoring.
- Permission-aware retrieval.
- Structured tool argument validation.
- Duplicate-action prevention.
- Defined escalation rules.
- Workflow-level tracing.
- Separate quality evaluation datasets.
Reliability review
During subsequent reviews, the engineering team can distinguish between failures associated with retrieval, model generation, external APIs, and business workflow logic.
This allows leadership to allocate resources according to measured failure patterns.
The example illustrates the value of defining reliability at the complete workflow level. It does not represent a verified customer deployment or measured improvement.
18. Common Mistakes When Setting LLM Reliability Targets
Mistake 1: Measuring Uptime Instead of Task Success
The model endpoint is available, but users receive incorrect or incomplete results.
Better approach: Measure technical availability and acceptable business outcomes independently.
Mistake 2: Selecting Arbitrary Reliability Percentages
Leadership demands 99.99% reliability without defining the measured event or examining operational requirements.
Better approach: Establish targets using user expectations, baseline evidence, business impact, and technical feasibility.
Mistake 3: Treating Every Failure Equally
A minor formatting issue receives the same classification as unauthorized information disclosure.
Better approach: Separate ordinary service failures from mandatory security and business constraints.
Mistake 4: Relying Exclusively on LLM Evaluators
Model-generated quality scores become the only evidence supporting production reliability.
Better approach: Combine deterministic assertions, domain evaluation, calibrated automated judges, and human review.
Mistake 5: Ignoring Workflow Dependencies
Component SLOs are reviewed independently without measuring their combined effect.
Better approach: Define an end-to-end workflow objective and use component metrics for diagnosis.
Mistake 6: Counting Every Fallback as Successful
The application escalates most requests to employees while continuing to report high reliability.
Better approach: Define acceptable fallback outcomes and separately monitor automation effectiveness and human intervention.
Mistake 7: Allowing Error Budgets to Justify Security Violations
An organization treats unauthorized actions as acceptable because they occur infrequently.
Better approach: Enforce mandatory security constraints independently of ordinary reliability budgets.
Mistake 8: Establishing SLOs Without Operational Ownership
Reliability objectives exist, but nobody is responsible for responding when they are breached.
Better approach: Assign accountable service owners, define escalation procedures, and connect reliability outcomes to engineering priorities.
Mistake 9: Ignoring Missing Evaluation Data
Unscored or unrecorded tasks are treated as successful by default.
Better approach: Track evaluation coverage, unknown outcomes, and measurement-system reliability.
Mistake 10: Treating Reliability Targets as Permanent
The application evolves while its objectives remain unchanged.
Better approach: Review objectives when business requirements, models, architecture, deployment scope, or user expectations change.
19. Production Reliability Checklist for LLM Workflows
Workflow definition
- [ ] Intended business outcomes are documented.
- [ ] Eligible requests are clearly defined.
- [ ] Successful and unsuccessful outcomes are distinguishable.
- [ ] Appropriate abstention and escalation behavior is defined.
Reliability objectives
- [ ] End-to-end task success is measured.
- [ ] Availability objectives are established.
- [ ] Quality and grounding requirements are documented.
- [ ] Execution correctness is evaluated.
- [ ] Latency objectives reflect user expectations.
- [ ] Recovery requirements are defined.
Security and governance
- [ ] Mandatory security constraints are documented.
- [ ] Authorization is enforced independently of model behavior.
- [ ] Critical violations trigger incident review.
- [ ] Ordinary error budgets do not override mandatory controls.
Error-budget management
- [ ] Measurement windows are defined.
- [ ] Error budgets use the correct measurement units.
- [ ] Budget consumption is calculated.
- [ ] Burn-rate alerts are configured where appropriate.
- [ ] Budget breaches trigger documented engineering decisions.
Observability
- [ ] End-to-end workflow traces are available.
- [ ] Component failures can be investigated.
- [ ] Evaluation results are linked to application versions.
- [ ] Missing and delayed telemetry is monitored.
- [ ] Sensitive logging is controlled.
Recovery and resilience
- [ ] External dependency failures are tested.
- [ ] Fallback behavior preserves applicable requirements.
- [ ] Retries are bounded.
- [ ] Consequential operations are protected against duplication.
- [ ] Rollback and recovery procedures are available.
Engineering operations
- [ ] Reliability ownership is assigned.
- [ ] Release policies consider reliability performance.
- [ ] Incident procedures are documented.
- [ ] Reliability improvements are prioritized using failure evidence.
- [ ] Objectives are reviewed periodically.
The completed checklist should be supported by actual measurements, evaluation evidence, operational records, and accountable approvals.
Frequently Asked Questions
What is a reliability target for an LLM workflow?
A reliability target defines the expected level of acceptable service behavior over a specified measurement window. It may cover task success, availability, response quality, execution correctness, latency, or recovery.
What is the most important reliability metric for an LLM application?
For many applications, end-to-end task success is the most useful primary metric because it measures whether the workflow achieves its intended business outcome. Additional metrics remain necessary for security, quality, and operational performance.
How are LLM reliability targets different from traditional software SLOs?
Traditional SLOs commonly focus on availability, request success, and latency. LLM workflows also require measurements covering semantic quality, grounding, model behavior, tool execution, and appropriate handling of uncertainty.
Should every enterprise AI application target 99.9% reliability?
No. The appropriate target depends on the measured behavior, business impact, user expectations, operational dependencies, and available engineering resources.
How do you calculate an error budget for an LLM workflow?
For a request-based SLO, multiply the total eligible requests by the permitted failure proportion. Error-budget consumption is the number of unsuccessful eligible requests divided by the permitted failures for the same measurement window.
Can AI answer quality be used as an SLO?
Yes, provided the quality indicator has a clear definition, representative measurement methodology, appropriate evaluation coverage, and documented limitations.
Can LLM-based evaluators determine workflow reliability?
They can contribute to semantic quality assessment, but deterministic tests, business-state validation, security checks, and human review may also be required.
How should engineering teams handle security violations within error budgets?
Mandatory security violations should be governed by independent incident and release policies. Ordinary reliability budgets should not authorize prohibited disclosures or unauthorized actions.
How do teams measure reliability in multi-step AI agents?
Teams should evaluate complete task outcomes, tool selection, argument validity, authorization, execution state, recovery behavior, and unintended side effects.
Can fallback responses count as successful tasks?
Yes, when the fallback satisfies the predefined workflow contract. Fallback frequency and automation effectiveness should also be measured separately.
What happens when an LLM workflow exhausts its error budget?
Engineering teams should follow an approved error-budget policy. This may involve restricting feature releases, prioritizing remediation, investigating failure patterns, and conducting controlled corrective deployments.
How often should AI reliability objectives be reviewed?
Objectives should be reviewed regularly and after material changes to models, prompts, retrieval configurations, integrations, business requirements, or deployment scope.
Conclusion: Define Reliability Around the Outcome Users Depend On
Production LLM applications require a broader understanding of reliability than conventional infrastructure availability.
A healthy model endpoint does not guarantee a correct answer.
A successful tool call does not guarantee a valid business operation.
A fast response does not guarantee an acceptable user experience.
Engineering leaders should establish reliability objectives across eight dimensions:
- End-to-end task success.
- Service availability.
- Response quality and grounding.
- Execution correctness.
- Latency and responsiveness.
- Security and policy enforcement.
- Recovery and resilience.
- Operational cost predictability.
These objectives should be supported by measurable indicators, error budgets, workflow observability, controlled release policies, and explicit operational ownership.
The next step is to select an existing LLM workflow and document what an acceptable outcome actually means.
Identify its dependencies, establish measurable indicators, evaluate current performance, and set targets that reflect the business consequences of failure.
Then use those targets to guide architecture decisions, release approvals, and reliability investment.
The purpose of an LLM reliability objective is not to prove that the model rarely fails. It is to establish whether the complete application can consistently deliver the outcome that users and businesses depend on.
來源:DEV Community · dev.to