技能不只是 Markdown 檔案：按 OWASP Agentic Skills Top 10 審查 scaffold-docs

A documentation skill sounds like a low-risk addition to an AI agent. It reads code, proposes an outline, and helps write documentation.

But those activities cross security boundaries. The skill reads repository content that someone else may control, carries context between sessions, and asks an agent to write files. Its safety depends on both its instructions and the permissions available to the agent.

We evaluated scaffold-docs, an existing documentation-writing skill, against the OWASP Agentic Skills Top 10. We then implemented a limited set of improvements and checked whether its original workflow still worked.

The most useful finding was not a hidden malicious command. It was how easily ordinary workflow instructions can leave trust, permissions, and completion state underspecified.

What we reviewed

The skill organizes documentation into three sections:

• Getting Started: a representative tutorial.
• Diving Deeper: explanations of concepts and design decisions.
• Reference: API documentation.

Its workflow is deliberately iterative. The user selects an audience, reviews the structure, and approves successive drafting passes. An update mode uses stored metadata to reconcile documentation after code changes.

We pinned the review to commit e234e13eb1ab9561a1da9a5a150c3deee3c515b8. That matters: a security assessment applies to the content reviewed, not automatically to whatever appears on the default branch tomorrow.

We also pinned the OWASP reference material and used its AST01–AST10 categories. This is the Agentic Skills Top 10, distinct from OWASP’s Agentic Applications taxonomy.

Start by treating the skill as evidence

We cloned the repositories into an isolated cloud workspace and inspected the target without installing it as an active skill.

That distinction was essential. A file under review may contain instructions addressed to an agent. Those instructions are evidence to analyze, not authorization to follow.

The review covered all 15 tracked files, including hidden metadata, phase instructions, bundled references, and historical implementation plans. We recorded file hashes and inspected file types and instruction content.

No intentionally malicious payload was identified in that snapshot. There were no bundled executable scripts or required remote instruction fetches.

That was a useful observation, but it was not a safety verdict. Natural-language instructions can still influence an agent’s actions without shipping executable code.

Use OWASP as a set of questions

We mapped the skill against all ten categories:

The framework helped organize the investigation. It did not turn every missing feature into a confirmed vulnerability.

For example, the absence of a permission manifest does not prove that a particular host grants excessive access. It means the deployment needs closer examination.

Separate observations from exploit claims

We combined manual instruction review with small, disposable fixtures.

One fixture tested the update workflow’s documented comparison:

git diff <baseline>..HEAD --stat

After modifying a tracked source file and adding an untracked API file, that comparison remained empty because neither change had been committed.

The command behaved correctly. The workflow’s scope was the problem: it did not clearly state whether it documented committed source or the current working tree.

Another fixture showed that a documentation filename could resolve through a symlink outside the intended directory. We did not write through that link.

These findings support different conclusions:

• The Git comparison demonstrably omits working-tree changes.
• The symlink fixture demonstrates a containment precondition.
• Neither test proves that an agent would perform an unauthorized write or escape its host sandbox.

That distinction keeps a security report actionable without overstating its evidence.

Be precise about isolation

The assessment ran in an isolated cloud task, but network access remained enabled for retrieving source material.

We also attempted a stronger nested Bubblewrap sandbox with network isolation. It failed during UID-map setup, before the child command ran.

We continued with static review and harmless local fixtures. We did not describe the assessment as air-gapped or claim that the additional sandbox boundary worked.

A useful rule emerged: report the boundary you verified, not the boundary you intended to create.

The biggest gaps were in trust and state

The review identified several improvement areas.

First, the skill asked the agent to read comments, examples, existing documentation, and audience metadata without explicitly separating those inputs from instructions that could authorize actions.

A stored audience description should influence tone and depth. It should not be able to grant tool permissions or skip a checkpoint.

Second, file-access scope was undefined. Relative output names did not establish where writes were permitted, and editorial checkpoints did not enforce filesystem containment.

Third, installation guidance allowed copying or symlinking the skill directory without a package-wide integrity procedure. A reviewed skill can drift if its installed resources change.

Finally, synchronization metadata needed a clearer lifecycle. Further evaluation found that the creation workflow recorded a “last synced” commit during planning—before the documentation was complete.

An interrupted session could therefore leave behind a convincing-looking sync marker for unfinished work.

Implement a focused set of changes

We limited implementation to four recommendation groups:

• R1: explicit instruction/data boundaries and trusted approval recovery.
• R3: separate in-progress work from completed synchronization.
• R4: validate metadata and use a stable, explicit source revision.
• R7: resolve audience precedence and non-Git behavior.

The modified workflow now keeps documentation in progress until the selected buckets are complete or explicitly excluded. Updates retain the previous completed baseline while new work is pending.

Git mode documents a captured committed snapshot. Dirty and untracked source is disclosed as excluded, and source reads use the captured commit consistently.

Audience handling also became deterministic: an existing audience remains active even when the user has disabled further audience prompts.

We explicitly left R2, R5, R6, and R8 unimplemented. Those cover broader filesystem enforcement, installation integrity, adversarial regression infrastructure, and runtime support or governance. Adding instructions does not complete those controls.

Functional validation caught a regression

Security improvements can break normal workflows.

During validation, we found that phase-entry checks rejected a completed record whenever HEAD had advanced. But a new commit is exactly what normally triggers a documentation update.

We corrected the distinction:

• State validation permits a completed record to reference the previous target.
• Starting an update captures the new target.
• Active source reads and completion still reject an unexpected HEAD change during the session.

After that correction, 30 checks passed. They covered state transitions, committed-source reads, interrupted work, updates, non-Git completion, and preservation of the original editorial instructions.

The original prose, headline, audience, and topic-selection references remained unchanged. The drafting passes and review checkpoints were preserved.

These were deterministic checks with simulated documents and approval outcomes. They did not establish end-to-end writing quality, real human-review behavior, or universal resistance to prompt injection.

Lessons for skill authors and reviewers

Review prose as part of the attack surface.

A Markdown-only package can still direct consequential tool use.

Keep editorial preferences separate from authority.

Audience descriptions, comments, and prior documents should guide content—not permissions.

Model completion honestly.

“Planning started,” “draft approved,” and “documentation synchronized” are different states.

Choose a source boundary and use it consistently.

Comparing committed changes while reading uncommitted files produces an ambiguous result.

Distinguish skill guidance from host enforcement.

A rule against writing outside a directory is useful. A host that denies the write provides a stronger boundary.

Test the ordinary path after hardening.

The update-entry regression was found through functional validation, not through searching for malicious payloads.

The resulting vetting repository contains the original assessment, recommendations, scoped implementation, and validation evidence. Its findings describe what was observed, what was changed, and what still needs testing—the distinctions a useful skill-security review should preserve.