無人值守的軟件工廠：哪些環節可自動運行，哪些需要人工把關

The tenth most-read story on daily.dev today is a question about the industry's newest industrial metaphor: are "dark software factories" the next big thing? The piece frames a future where AI agents write, test, deploy and monitor software with nobody watching — software's version of a lights-out factory. It is the hottest conversation in the AI-agents corner of the feed right now, and for once the hype question is the wrong question. The useful question is not "can agents run a factory" — it is which stations on the line can actually run dark, and which ones need a human standing at the gate.

We run automation that operates unattended for days: scraping pipelines, agent-driven rollouts, validation systems. The difference between a pipeline that survives production and one that quietly rots is never "more AI". It is the placement of the humans. This is the practitioner version of the dark-factory debate: what runs lights-out, what breaks, and the checkpoint design we actually use. It follows the same philosophy as how we work — automate the routine aggressively, and spend human judgment exactly where it changes the outcome.

1. The borrowed term, and what the borrowing gets wrong

A "dark factory" is a manufacturing concept: a plant that runs without human presence, lights out, because the work is standardized enough for machines. The borrowing is flattering to software because it implies software work is standardizable in the same way.

It is not. A CNC mill receives standardized inputs — raw stock of known dimensions — and its outputs are measurable against a tolerance in microns. Failure modes are bounded and enumerated in advance. Software work has none of those properties:

• Inputs are adversarial and non-standard. Requirements arrive as prose. Dependencies update themselves. Third-party targets change their headers, rotate their fingerprints, and deploy new bot detection on a Tuesday (we have the scars: our Shopee security research documented a full anti-bot stack that shifts under you).
• Outputs are hard to verify. A mill measures the part; an agent ships a diff that "passes CI" while subtly changing behavior. Verification in software is usually another program with its own blind spots — not a caliper.
• The factory floor fights back. Manufacturing robots do not face an adversary. Scraping, deliverability, fraud detection and security do. An unattended pipeline in an adversarial environment does not degrade gracefully; it degrades silently.

None of this means lights-out is a fantasy. It means "dark" is not a property of the factory — it is a property of each station. Some stations are dark-ready. Others break specifically because nobody is there.

2. What actually runs lights-out today (and why)

Look at the automation that genuinely runs unattended in production, and a pattern emerges. A station can go dark when it satisfies three properties at once:

1. Bounded inputs. The agent's decision space is finite and enumerable.
2. Cheap, independent verification. You can check the result without the same judgment that produced it.
3. Reversible failure. When it goes wrong, you can roll back and the world is as it was.

Concrete stations that satisfy all three:

• Deterministic transforms. Formatting, linting, code generation from schemas (OpenAPI → clients, protobuf → stubs). The input is a spec; the verifier is a compiler or a diff against the previous generation. This is dark today and has been for years.
• Dependency-bump proposals. An agent opens the pull request bumping a library; the gate is the existing test suite plus a policy check on the version range. The agent never merges — the gate decides. Reversible by revert.
• Metric-gated rollouts. The deployment pipeline from our PHP2Shell patch playbook: canary, health signals, automatic rollback. The rollout executes unattended; the decision to continue is a verification function with thresholds, not a person. When the threshold trips, a human is paged — the factory has lights at the exact station where darkness becomes expensive.

Notice the shape: in every genuinely dark station, an agent does the doing and a separate, dumber mechanism does the verifying. The moment the doer and the verifier are the same mind, darkness becomes optimism.

3. The failure taxonomy: where lights-out breaks

Three failure modes recur in unattended automation, and each one has a specific human-shaped hole.

Grounding gaps: the proxy is not the thing. Agents optimize whatever signal you give them. Give them "all tests pass" and you get green tests, not correct behavior. This is not a theoretical alignment problem — it is a Tuesday problem. In our email-validation work, an SMTP server that answers "yes" to a recipient probe is not ground truth: catch-all servers accept everything, so the probe's "valid" verdict is a lie told politely (we documented the whole taxonomy). Every dark station inherits this risk: the verifier must be grounded in a signal the agent cannot game. If your gate and your generator share the same LLM, they share the same hallucinations — errors correlate, and correlated errors are the ones that compound.

Adversarial drift: the floor moves. An unattended pipeline in a contested environment rots. We watched scraper fleets run green for months, then die overnight when a target changed one header and started scoring TLS fingerprints differently. The automation was correct; the world moved. Lights-out works where the world is stationary. Where the world is adversarial — bot detection, fraud, deliverability — the pipeline needs a feedback station that measures outcome (did the data arrive correct? did the mail land?), not just execution (did the job run?). Drift detection is the station that keeps the factory's lights on.

Irreversibility: blast radius is the real risk metric. The question at every station is not "can the agent do this" but "what happens if it is wrong". Dropping a database table, merging a schema migration, emailing ten thousand leads, deploying to a payment path — the dark factory's catastrophes all come from letting an agent perform an irreversible action on an unverified belief. Reversibility, not capability, should decide what goes dark.

4. The checkpoint design: gates keyed to blast radius

The practical answer is not "human reviews everything" (that is the pre-factory era) and not "agents do everything" (that is the accident). It is a gate: every automated action carries a declared blast radius and reversibility, and the gate routes it — auto-approve, auto-approve-with-rollback, or require-human. This is the same human-in-the-loop checkpoint design we consider core to automation that survives contact with production.

A minimal version, in the shape we actually implement for pipeline actions:

from dataclasses import dataclass
from enum import Enum

class BlastRadius(Enum):
    LOW = "low"          # reversible, single-record, retry-safe
    MEDIUM = "medium"    # bulk but reversible within 24h
    HIGH = "high"        # irreversible or customer-facing

@dataclass
class Action:
    name: str
    blast: BlastRadius
    reversible: bool
    undo: callable = None   # must exist if reversible

def gate(action: Action, verifier) -> str:
    """Route an automated action. Returns 'auto', 'auto+watch', or 'human'."""
    # 1. Claim before work: idempotency first — a gate that can't tell
    # "already done" from "do it again" is not a gate, it's a gun.
    if verifier.already_done(action):
        return "skip"

    # 2. Irreversible or high blast radius: a human signs, no exceptions.
    if action.blast == BlastRadius.HIGH or not action.reversible:
        return "human"

    # 3. Medium: run, but watch. Verification is a separate, dumber
    # mechanism than the agent that proposed the action — outcome
    # signals, not "it says it worked".
    if action.blast == BlastRadius.MEDIUM:
        return "auto+watch"

    # 4. Low + reversible + idempotent: genuinely dark.
    return "auto"

def run(action: Action, verifier):
    decision = gate(action, verifier)
    if decision == "skip":
        return {"status": "skipped", "reason": "idempotent claim"}
    if decision == "human":
        return {"status": "queued_for_human", "action": action.name}
    result = action.execute()
    # Auto-rollback on outcome failure, not on self-report.
    if not verifier.outcome_ok(action, result):
        if action.reversible and action.undo:
            action.undo()
        return {"status": "rolled_back", "action": action.name}
    if decision == "auto+watch":
        verifier.register_watch(action, window="24h")
    return {"status": "done", "action": action.name}

Four properties make this more than pseudocode optimism:

• Idempotency is the first gate (already_done). A station that cannot distinguish "already done" from "do it again" will double-charge, double-send and double-deploy. Idempotency keys and claim-before-work are the precondition of unattended execution, not a nice-to-have.
• The verifier is dumber than the agent on purpose. outcome_ok checks outcomes — the data arrived, the record matches, the canary metrics hold — using signals the proposing agent cannot produce. If the same model both proposes and judges, you have theater with two acts.
• auto+watch is the middle that most designs skip. Medium-blast actions run but register a 24-hour outcome watch. Drift detection lives here: did the mail actually land, did the scrape return the same schema, did the fingerprint still verify? This is the station that catches adversarial drift while the factory runs dark.
• The human queue is an asset, not a failure. queued_for_human is a designed output. A factory where nobody is ever queued is a factory whose blast radius is capped by luck.

This pattern scales precisely because the human is a routed exception, not a bottleneck: in a healthy week, the queue is near-empty, and every item in it is there because an irreversible or adversarial decision needed judgment.

5. The lights-out scorecard

Apply the taxonomy station by station and you get a working map. "Dark" is a verdict per station, earned on three axes: cheap independent verification, reversibility, and absence of an adversary.

The last row is worth pausing on, because it is the closest thing to a real dark factory most teams will operate this year: when an emergency lands, automation finds every runtime, bumps every version, restarts every worker and verifies every host — the doing is dark, the plan is human, and verification is outcome-based. That is the shape that works, and it generalizes.

6. Where the debate should go next

The dark-factory conversation will mature the way every automation conversation matures: from "will agents replace X" to "what is the gate budget". A gate budget is the honest accounting of a pipeline: how many human judgments per hundred automated actions, and at which stations. Teams that run real unattended automation already keep this ledger informally — the on-call rotation is a gate budget with a pager.

Three predictions from the floor, not the balcony:

1. The winning unit is the gated agent, not the autonomous one. Frameworks will compete on how well they declare blast radius, enforce idempotency, and route to humans — the boring infrastructure of the gate() function above.
2. Drift detection becomes a product category. In adversarial environments, "did the world move" is a first-class signal, and pipelines that cannot answer it are not dark — they are blind.
3. The human queue gets a UI. Right now it is Slack pings and dread. Mature dark factories will treat the human-review queue like a work queue: prioritized, SLAs, audit trail — the station with the lights on, designed as carefully as any dark one.

The factory can be as dark as its gates are good. Build the gates first, earn the darkness station by station, and keep one light on — exactly where the blast radius says it belongs.

Frequently asked questions

Is a "dark software factory" just hype for "CI/CD with agents"?
Mostly, today. The new part is agents proposing changes across the whole lifecycle, not just running a fixed pipeline. The discipline that makes it work — gates, idempotency, outcome verification — is the same discipline CI/CD taught us; the blast radius is larger, so the gates must be better.

Can tests be the verifier for agent-written code?
Only if the tests are not written by the same mind that wrote the code. An agent that writes code and its tests, then reports "all green", has optimized a proxy. Independent verification means a signal the generator cannot produce: existing golden tests, property-based checks against a spec, production shadow traffic, or a human.

What is the single cheapest gate to add to an existing pipeline?
Idempotency claims before every side effect. Most "the agent did it twice" incidents die at this gate, and it costs a key and a lookup per action.

Should small teams bother with this, or is it enterprise theater?
Small teams feel the failure modes first, because one person is the entire human queue. The gate pattern above scales down well: the point is routing the three irreversible decisions a week to yourself instead of reviewing everything.

Does adversarial drift mean scraping can never go dark?
No — it means scraping goes dark with a drift watch, not dark forever. Outcome signals (schema stability, volume bands, spot-check verification) are the lights that stay on while the station runs unattended.

References

• daily.dev Popular feed — trend signal for this piece (top-10 post on AI-agent "dark software factories", 2026-10-23): https://daily.dev/posts?sortBy=popular
• Botslave — PHP2Shell patch-operations playbook (dark execution, human-owned plan): https://botslave.com/blog/php2shell-patch-operations-playbook
• Botslave — Git workflows for AI agent teams (reviewable, revertable rollouts): https://botslave.com/blog/git-for-ai-agent-teams
• Botslave — Email validation and why SMTP probing lies (grounding gaps): https://botslave.com/blog/email-validation-mx-catch-all-smtp-probing
• Botslave — Shopee security headers and bot detection (adversarial drift): https://botslave.com/blog/shopee-security-headers-tls-tokens-bot-detection

Originally published on botslave.com. Botslave is an automation studio — see what we build and how we work.