跳到正文
DEV Community· AI Stack Lab·· 1 小時前AI 評分53

RulePort 評測:Agent 規則包的可靠程度取決於安裝器

Your agent's rule pack is only as good as its install script

AI 導讀

AI Stack Lab 以六種情境實測自家 RulePort 安裝器,四項主張成立,另發現兩項使用上的落差。漂移檢查在零檔案比較時仍顯示同步並以 exit 0 結束,作者建議核對 comparing 12 file(s) 的數量。

正文

This is the adapted syndication of a longer review first published on AI Stack Lab: RulePort Review: I Ran Its Installer Six Ways. The blogger version has the full command-by-command transcript.


Your agent's rule pack is only as good as its install script

Every "rule pack" sold to people running two or more coding agents makes the same three promises: one source of truth, one installer, and a drift check that tells you when the files on disk no longer match what you bought. The first promise is an authoring choice. The second and third are code, and code can be tested.

So I tested them. I unzipped RulePort ($1 on Gumroad, 58 files, 12 rules) into an empty directory, copied it to a scratch tree, and ran its installer six ways: dry-run, real, twice to check idempotency, over hand-written files, on a machine with no agents at all, and against deliberate drift. Then I hashed the trees before and after each run so "unchanged" meant a matching sha256, not a hopeful reading of the console.

Four claims held. Two leaked. Here is what that looked like.

The parts that held

The compiler is byte-stable. python3 tools/ruleport.py compile printed compiled 12 rules -> 39 files. I hashed the shipped compiled/ tree, compiled, hashed again, compiled a third time, hashed again. Zero files changed in either direction. This sounds like a low bar and it is not: a compiler that reorders its own output every run makes every commit dirty and turns the drift check into noise. This one is deterministic.

--dry-run actually writes nothing. Not "writes then tells you" — I hashed the target directory before and after and it was empty.

Install is idempotent. First run: 39 files. Second run, same command: every file reported unchanged, and the sha256 map of the whole tree was identical. No duplication, no second marker comment, no churn.

It refuses to eat your own files. I planted a hand-written CLAUDE.md and AGENTS.md with no RulePort marker and ran the full install. Both were skipped and left byte-identical. --force replaced them deliberately, as documented. That default matters more than the feature list: an installer that silently overwrites a CLAUDE.md you wrote is worse than no installer at all.

The fallback fires. On a fresh HOME with agent binaries stripped out of PATH, install.sh printed no runtimes detected and wrote exactly one file: ./AGENTS.md. The README's claim about the universal fallback is true on a machine with nothing installed.

diff means what an exit code should mean — mostly. Fresh install: in sync — installed files match the compiled pack, exit 0. Append one line to a single .mdc rule and re-run: exit 1, with a real unified diff naming the file. Bad arguments exit 2 with usage. I will take a tool that shouts over one that guesses.

The two leaks

Leak one: diff can report "in sync" over zero files. Run it in a directory where nothing was ever installed and it prints one not installed … — skipped line per runtime, then in sync — installed files match the compiled pack and exit 0. Four runtimes skipped, zero files compared, green tick.

If you follow the README and wire that exit code into CI as your drift gate, a machine where the install never happened passes. You have to read the comparing N file(s) lines, or assert on a file count you expect, instead of trusting the number.

I want to be fair about this: a drift check with nothing to compare arguably is in sync with an empty installed set. But that is not the sentence anyone reads at 2am when the CI is green and the agent is still following last month's rules. A gate that can pass by comparing nothing is exactly the false confidence this class of tool exists to remove.

Leak two: "claude detected" means a claude binary is on PATH. Detection is [ -d "$HOME/.claude" ] || command -v claude. On my box that fired and the installer wrote 14 files into ~/.claude — globally, from a project directory, without asking. The README's table says Claude Code lands in ~/.claude/ (global) or .claude/ (per project), but bash install.sh only ever does the global one. To get a project-local .claude/ you must call python3 tools/ruleport.py install --agent claude --target <dir> directly. I confirmed both by doing both.

Neither leak is dishonest — the behaviour is documented loosely enough to be defensible. But "global by default" plus "green drift check that compared nothing" is a pair of surprises worth knowing before you buy, so they belong in the review, not in a footnote.

What is in the box, counted

  • 58 files, 174,768 bytes unpacked, 103,003 bytes as the zip.
  • 12 canonical rule files — the only place you edit.
  • 39 compiled outputs: 12 Cursor .mdc, 14 Claude Code files (12 skills + CLAUDE.md + a /ruleport-status command), 12 Windsurf, 1 AGENTS.md (24,336 bytes, all 12 rules).
  • tools/ruleport.py: 457 lines, imports only argparse, difflib, pathlib, shutil, sys. Standard library, so there is no dependency graph to audit and no network call at compile time.
  • install.sh: 111 lines. Plus docs/customization.md and docs/update-policy.md.

The 12 rules are deliberately boring: verify before done, test first, error handling, types and docstrings, security baseline, self code review, database safety, API design, dependency hygiene, git hygiene, frontend accessibility, docs updates. They are the ones that stop the classic agent failure — the confident "done" over untested code, the swallowed exception, the dependency added for a problem the standard library already solved.

Who should skip it

  • You run exactly one agent. A compiler that emits four formats is pure overhead for one consumer. Hand-write your rules.
  • Your team already maintains a conventions tree that agents read. You have a source of truth; a second one is a liability.
  • You need a drift gate you can trust blind. Because of leak one, you must assert on the compared-file count, not the exit code alone.
  • You want rules generated from source. This ships opinionated prose, not a linter and not a static analyser.
  • You need Windows-native tooling. The compiler is portable Python; the installer is bash.
  • You expect the four runtimes to behave identically once installed. A .mdc glob is not a Skill trigger. RulePort standardises the text, not the loaders.

Honesty block

Example figures are illustrative planning inputs — not market rates and not earnings. Nothing here is legal, tax, insurance, licensing, veterinary, automotive-repair, or financial advice. No claims are made about CRM, map, API, or invoicing integrations that this bundle does not ship — it ships 12 canonical rule files, one 457-line Python compiler, one 111-line bash installer, and two docs, as counted above. No ratings, reviews, testimonials, or sales figures are quoted here, because I have none that would be honest to quote; the live listing shows a rating count of zero.

Disclosure: this is our own product, not an affiliate placement — there is no third-party commission. Price verified on the live page at the time of publishing: $1.00 USD (the page also renders a local-currency estimate). Link: https://nandminc.gumroad.com/l/xcbqxr

Two habits I changed after measuring it, both one line: install with an explicit --target when project scope matters, and assert on comparing 12 file(s) rather than a green exit code. Which is, more or less, what the pack preaches about verification generally.

來源:DEV Community · dev.to