Don't Trust a Skill by Its Description

You are about to install a third-party skill because the description sounded right. That description is already a selector in the system prompt. The body loads on trigger. Bundled scripts can run with only stdout in chat. Before first use, open the whole tree, pin a content hash of what you reviewed (not the name or blurb), skip auto-load for untrusted skills, and re-hash after the first run. Same instinct as binding MCP approval to content; different artifact.

Field rule

Open the full skill tree before first use: frontmatter, body, scripts/, references/, and assets/. Treat that tree as instructions that inherit your agent's tools. Bind install and load to a content hash of the bytes you reviewed, not the skill name or description. Prefer allowlisted publishers. Do not auto-load untrusted skills from description alone. After the first run, and after any update, re-hash. If the digest moved, treat it as a new consent event.

You liked the description. What actually gets loaded?

The registry card reads clean. One-line description matches the job. You install.

Here is the part the install UI usually understates. Under the open Agent Skills model, every installed skill puts its name and description into the system prompt at startup (on the order of ~100 tokens each). When the agent decides that description matches the user's request, it loads the full SKILL.md body into context. Bundled scripts/, references/, and assets/ come in only as needed. Scripts can execute so that only stdout lands in the chat. A reviewer who only reads the conversation never sees the payload. [1][2][3]

So the useful question is not "does this sound helpful?" It is: what exact instruction tree just inherited your tools?

Walk the tree before first use

Treat the skill directory the way you would treat a third-party package you are about to run with your credentials nearby. Open these layers in order.

1. Frontmatter (name + description). These are not marketing copy for humans alone. They are the match key. Startup loads them for every installed skill. The agent uses the description to decide when to pull the rest. If you only skim the registry blurb, you are reviewing the selector, not the behavior. [1][2][3]

2. The SKILL.md body. This is the prompt. The open spec places no format restrictions on that markdown. The validator checks frontmatter. It does not pin content, sign the package, or force re-approval when the tree changes. [1][2]

3. scripts/. Assume a script can run without its source appearing in chat. Read every script the way you would read an install hook. Look for network calls, env access, credential paths, and anything that shells out. [3]

4. references/ and assets/. Templates, examples, configs, and linked docs are part of the instruction surface. Camouflaged attacks often live here as "helpful" code samples. Open them before first use, not after something weird happens.

5. Product knobs that look like sandboxes but aren't. Claude Code's allowed-tools is a turn-scoped grant, not a hard deny-list for other tools. Live reload of SKILL.md edits in-session is documented. Neither is a content-integrity control. [4]

If your product supports disable-model-invocation or slash-only invocation for untrusted skills, use it until you have finished this review. [4]

Why the description is a bad trust signal

Registry rank and a polished description feel like social proof. They are also an attack surface.

Saha, Faghih, and Feizi (arXiv:2605.11418, preprint, not peer-reviewed) study a registry adversary who can change only SKILL.md text: no code, no aux files, no registry backend. Discovery attacks reach an 86.14% pairwise win rate and 80% Top-10 placement against OpenAI text-embedding-3-small. On a ClawHub-style ranking replica (n=100 skills), avg-day win rate is 74.14%. A 0-day skill hits 94% at t=0 and 40% after one hour. Description-only adversarial variants win selection in 77.6% of paired trials on average. Governance evasion (avoid a block) spans 36.5% to 100%. Context-window overflow labeled 87.1% of cases clean and 0% malicious. Truncating review to the first 10K characters is an attack surface, not a control. [5]

The authors' point for builders is simple. SKILL.md is operational text that shapes which third-party capabilities agents find, trust, and use. Ranking is not evidence of safety.

What can hide after you install

Once the skill is on disk, the problem shifts from "will it get selected?" to "what does the body teach the agent to do?"

Qu et al. (arXiv:2604.03081, preprint) assume retrieval already succeeded. They poison the skill body with DDIPE: malicious logic dressed as benign code examples and config templates. From 81 seeds they build 1,070 adversarial skills across 15 MITRE categories. End-to-end bypass rates run 11.6% to 33.5% (n=1,070). Explicit-instruction attacks drop to 0% under their strong-defense setup. Scanners catch shouting. They miss camouflage. Static SkillScan intercepts 90.7%. Still, 2.5% (27/1070) evade static analysis and alignment. Cross-model joint bypass on Claude Code: 1.6%. Four vulns disclosed; two fixed. [6]

Wild registries show the same class of problem in separate snapshots. Do not merge them into one "current" count.

Snyk's ToxicSkills scan as of 5 Feb 2026 covered 3,984 skills on ClawHub + skills.sh. From the body table (ignore the page H1): 1,467 (36.82%) had any security issue; 534 (13.4%) had at least one critical; 76 HITL-confirmed malicious payloads; 8 still listed live on ClawHub as of that publication (IOCs not re-checked on 2026-09-14). At research time Snyk described the publish barrier as a SKILL.md plus a roughly one-week-old GitHub account: no signing, no review, no default sandbox. Payload shape was often natural-language setup instructions that steer the agent, not a classic binary in the registry entry. [7]

Separately, Trend Micro reported 39 OpenClaw skills used to push Atomic macOS Stealer, also found on skills.sh / SkillsMP. [8]

A later ClawHub Security Signals export (31 May 2026) covered 67,453 latest public versions: ClawScan 61.9% clean / 37.8% suspicious / 0.3% malicious (206). Those are automated silver-standard labels, not human ground truth. SkillSpector findings are advisory and do not alone block install. [9][10]

skills.sh's audit board is useful telemetry. The docs state an explicit no guarantee on every skill. [14][15]

One install review is not enough

Ning et al., SkillHarm (arXiv:2606.02540, preprint): 879 samples across 71 skills and 12 risk types. Two scenarios matter on the desk.

Fixed-Payload Poisoning (FPP) harms in the same session. Self-Mutating Poisoning (SMP) rewrites persistent skill content on first run; harm shows up on reuse. Peak ASR: FPP 86.3%, SMP 69.3% (Codex / GPT-5.4). Claude Code / Opus 4.7: FPP 27.4%, SMP 9.4%. Conditional ASR jumps +32.1 points on SMP when you stop counting "never opened the file" as a defense. Scanners top out at 55.6% FPP / 68.8% SMP detect. Defensive system prompts: nearly all configs still >70% ASR. [11]

That is why a green install review is not a permanent consent. Claude Code also documents live reload of SKILL.md edits in-session. [4] If the bytes can change after you reviewed them, your approval has to follow the bytes.

A 15-minute review ritual

Do this before first use of any third-party skill you did not write yourself.

1. Write down the install record. Path on disk, publisher or marketplace, why you want it, who approved it. Inventory skills the same way you inventory MCP servers.

2. Open the whole tree. Frontmatter, body, every file under scripts/, references/, and assets/. Search for curl, network hosts, env reads, hooks, credential paths, and "run this setup" language. Assume scripts may run without appearing in chat.

3. Pin a content hash of the skill directory (canonical file set: sorted paths, normalized line endings if that is how you canonicalize). Store the digest next to the install record. The Cloud Security Alliance's May 2026 note names content-hash verification before production load explicitly for skills. [12]

4. Turn off auto-load for untrusted skills where the product supports it (disable-model-invocation / slash-only). Description matching is how the skill enters context. You do not owe every installed description a free ticket into the system prompt. [4]

5. Allowlist publishers and marketplaces. Claude Code has strictKnownMarketplaces and plugin allowlists. Registry rank is not a substitute. [13]

6. Shrink the blast radius outside the model. No extra tools, no secrets in the skill's env, default-deny egress. Do not treat allowed-tools as that control. Spec: experimental. Claude Code: turn-scoped grant, not a deny-list. [1][4]

7. Run once in a tight environment, then re-hash. After first use (and after any update), recompute the directory digest. If it differs, stop. Diff the tree. Treat the change as a new consent event. Pair this with a denied-path smoke test the same way you would for agent capability diffs.

Vendor posture today is still mostly "audit it like software": trusted sources, read SKILL.md and scripts, marketplace allowlists. Fetched Claude / agentskills docs do not document cryptographic pin of the skill body plus forced re-approval on hash drift. That control has to live in your install path. [2][3][4][12]

What fails if you skip the ritual: reviewing only the name or blurb (description-only selection wins 77.6% of paired trials in 2605); treating SKILL.md as a README; trusting scanners or SkillSpector as a hard gate; leaning on a defensive system prompt; reading only the chat. [1][3][5][9][11]

Related surfaces on this site (same "text that runs with tools" problem, different artifacts): MCP tool poisoning, MCP prompts at the tool boundary, treat GitHub issues as agent input.

Same instinct as the MCP pin

Last week the object was the MCP config and tool definition: bind approval to content, not the tool name. This week the object is the skill directory.

Approval follows the bytes you reviewed, not the familiar label. A skill can rank itself into view with a polished description, hide payload inside an example, and rewrite itself after first use. The control is the same family: pin what you reviewed, re-consent on drift.

If you install from our hub, take the ritual with you. Full-tree review, content hash, no silent reload after writeback. Start at /skills/.

Sources

1. Agent Skills Specification - https://agentskills.io/specification
2. Anthropic Engineering, "Equipping agents for the real world with Agent Skills" - https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills
3. Claude Platform docs, "Agent Skills overview" - https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview
4. Claude Code docs, "Extend Claude with skills" - https://code.claude.com/docs/en/skills
5. Saha, Faghih, Feizi, "Under the Hood of SKILL.md: Semantic Supply-chain Attacks on AI Agent Skill Registry," arXiv:2605.11418 (preprint, submitted 12 May 2026) - https://arxiv.org/abs/2605.11418
6. Qu et al., "Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems," arXiv:2604.03081 (preprint, submitted 3 Apr 2026) - https://arxiv.org/abs/2604.03081
7. Snyk, "ToxicSkills: Malicious AI Agent Skills on ClawHub" (corpus as of 5 Feb 2026) - https://snyk.io/blog/toxicskills-malicious-ai-agent-skills-clawhub/
8. Trend Micro, "Malicious OpenClaw Skills Used to Distribute Atomic macOS Stealer" - https://www.trendmicro.com/en_us/research/26/b/openclaw-skills-used-to-distribute-atomic-macos-stealer.html
9. Hugging Face / OpenClaw, "ClawHub Security Signals" (snapshot 31 May 2026) - https://huggingface.co/blog/OpenClaw/clawhub-security-signals
10. OpenClaw, "OpenClaw × NVIDIA skill security" - https://openclaw.ai/blog/openclaw-nvidia-skill-security
11. Ning et al., "SkillHarm: Lifecycle-Aware Skill-Based Attacks via Automated Construction," arXiv:2606.02540 (preprint, submitted 1 Jun 2026) - https://arxiv.org/abs/2606.02540
12. Cloud Security Alliance, "SKILL.md / Agent Context Poisoning" research note (6 May 2026) - https://labs.cloudsecurityalliance.org/wp-content/uploads/2026/05/CSA_research_note_SKILL_md_agent_context_poisoning_20260506-csa-styled-1.pdf
13. Claude Code docs, settings reference (marketplace / plugin allowlists) - https://code.claude.com/docs/en/settings-reference
14. skills.sh docs - https://www.skills.sh/docs
15. skills.sh audits - https://www.skills.sh/audits