aitmpl.com 接入 NVIDIA SkillEvaluator,顯示六種智能體的技能載入狀態
NVIDIA SkillEvaluator shows which of six agents actually load your skill on aitmpl.com
aitmpl.com 接入 NVIDIA SkillEvaluator,新增 Compatibility 分頁,逐一顯示六種智能體會完整載入、降級載入或忽略技能。
I wired NVIDIA SkillEvaluator into aitmpl.com this week. Every skill page now has a Compatibility tab that says, agent by agent, whether Claude Code, Codex, Copilot CLI, OpenCode, Hermes and OpenClaw would load that skill, load part of it, or ignore it.
That is a different question from the one a security scan or a quality score answers. Those assume the skill reached the model. This one asks whether it ever got there.
What the Harness Check answers
A skill can follow the Agent Skills specification exactly and still never reach the model. Every harness has its own loader: its own YAML parser, its own folders, its own size limits. The same SKILL.md can work in Claude Code and be dropped by Codex with no error anywhere.
Each agent gets one of three verdicts:
-
loads: no loading or degradation rule matched. -
degraded: it loads, but something is cut, hidden or ignored. -
not_loaded: the harness drops it and the model never sees it.
The rules come from versioned profiles that ship with SkillEvaluator, built by running each harness and capturing the request it actually sent to the model. So a rule describes what the harness did, not what its documentation says, and each profile records the release and date it was verified against.
The skill in that screenshot is devil, and it is a good example. Copilot CLI drops it because the description is 1,178 UTF-16 units, 154 over the 1,024 limit, and an emoji counts as two. Codex keeps the skill but cuts the description to 1,021 characters in the skill list, which is exactly the text the model uses to decide whether to call it.
Run it on your own skill
The check is a CLI, so you do not need the catalog to use it:
pip install "skillevaluator==1.0.1"
skillevaluator harness-check ./my-skill
skillevaluator harness-check ./my-skill --harness codex,claude-code -v
skillevaluator harness-check ./my-skill --json harness.json
It takes a skill folder, a folder of skills, or a plugin folder, and it exits 0 whatever it finds. It is also part of Tier 1 of skillevaluator validate, so skillevaluator validate ./my-skill --tiers 1 runs it alongside the quality, schema, PII, license and security checks.
Four flags describe how you actually run the agent, and they change the limits:
-
--context-tokens N: your model's context window. -
--custom-provider: your model is served through a custom or OpenAI compatible provider. -
--explicit-invocation: skills are activated by an explicit mention, such as Codex$skill. -
--headless: the harness runs non interactively, such asclaude -pin CI.
Without them it assumes an unknown context window, a native provider, a model that opens the skill on its own, and an interactive session.
What 945 skills look like when you check all of them
The catalog run covers 945 skills against 6 agents, so 5,670 skill and agent pairs. Of those, 5,466 load as written, 203 load degraded and exactly 1 is dropped. By skill, 807 are clean in all six and 138 are not.
The non loading results are not spread evenly:
OpenClaw 132 119 of them: SKILL.md over the read cap
OpenCode 22
Codex 19
Hermes 15
Copilot CLI 12
Claude Code 4 all of them: price-like $N read as an argument
Two families cover almost everything. The first is size: OpenClaw counts each CJK character as four or more, so a long SKILL.md crosses its read cap long before you expect it to. The second is placeholders. $ARGUMENTS and ${CLAUDE_SKILL_DIR} are real Claude Code substitutions, and the other five agents hand them to the model as literal text. Claude Code's own four findings are the mirror image: a price like $4.00 in the body is read as an argument placeholder.
None of that shows up as a broken skill. It shows up as a skill that quietly does less than you wrote.
What it does not do
It does not gate anything. In the Claude Code Templates repo the job lives in skill-security-scan.yml, runs on skills changed in a pull request, and is marked continue-on-error, so it posts a report and never fails the PR. The comment goes on same repository pull requests; fork PRs get the result in the job summary instead.
On the site it is display only. The script that attaches the data says so plainly: the signal never changes ordering, search or ranking. Results are cached by file hash and SkillEvaluator version, so a catalog build only rechecks skills that actually changed.
And it does not replace running the agent. It predicts loader behaviour from profiles, which means it can tell you a skill will be dropped, but not whether it helps once it is loaded. That is a live evaluation, which is Tier 3 and needs a provider, an agent and Docker.
The pairing is the useful part. SkillSpector answers whether a skill is safe to install. The Harness Check answers whether the agent will read it. A skill needs both to be worth publishing.
Where the skills live
- The skill from the screenshot: devil
- A skill that passes all six: game-development
- The full skills catalog: aitmpl.com/skills
- Everything else: aitmpl.com
- NVIDIA SkillEvaluator: the repo and the Harness Check docs
You can follow me on my social media:
- Twitter (English): https://x.com/dani_avila7
- LinkedIn (Spanish): https://www.linkedin.com/in/daniel-avila-arias/
- Youtube (Spanish): https://www.youtube.com/@daniiielsan
- Instagram (Spanish): https://www.instagram.com/danielsan_avila/
來源:DEV Community · dev.to

