Building on Agent Skills: a field report.

Lund Studio · 9–10 October 2026. Written for Anthropic's leadership and the Agent Skills team. This report is independent; Anthropic has not reviewed or endorsed it.

Summary

Lund Studio is one person who sells AI skill cards: Agent Skills packaged for buyers. On 9 October 2026 we measured how often Claude, on Haiku, Sonnet and Opus through Claude Code, picks our skills when it should and leaves them alone when it should not. We rewrote the descriptions that failed, measured again, cut thirteen new cards, scanned real packets, and reviewed a page that writes skills. Three findings seem most useful to the platform:

1. Claude Code loads a SKILL.md that strict YAML readers refuse, so an author can ship a file that passes their own tests and fails elsewhere. We did.
2. When Claude passes over a skill, the author cannot see what it chose instead, and the answer differs by model.
3. Skills are a supply-chain surface. A published hash proves files match each other, not who made them or that anyone screened them.

1. What we measured

Our cards sat together in one Claude Code project, beside the descriptions of Anthropic's public PDF, PowerPoint, Word, Excel and frontend-design skills, standing in for what a buyer already has. In the trigger tests only names and descriptions were real. Each card got prompts a buyer would type, never naming it, and near-misses that belong to a sibling, a built-in, a clinician or nothing. Each run was a fresh claude -p session with file writes, the shell and the web blocked. We recorded the first skill Claude reached for, or "none". A hit is the right card on its own prompts; a wrong pick is a card used on someone else's.

Test	Runs	Result
First ten cards, as shipped	80 prompts × 2 × 3 models; 479 scored (6 timeouts left out; one Opus run never finished)	Right first choice: Haiku 112 of 160, Sonnet 132 of 160, Opus 125 of 159
Five of those lines, rewritten	42 prompts × 2 × 3; 249 scored	Haiku 71 of 84, Sonnet 73 of 84, Opus 76 of 81; no card picked on the 75 runs meant for none
One template, fixed (9 cards)	36 prompts × 1 × 3	Hits 39 of 81 → 78 of 81
One card's line, rewritten	12 prompts × 2 × 3	Hits 16 of 48 → 46 of 48
Thirteen new cards, cut to these rules	12 prompts × 2 × 3 each	First lines: 585 of 624 hits (Haiku 170 of 208, Sonnet 208 of 208, Opus 207 of 208), 3 wrong picks. After rewrites on six cards: 609 of 624 (Haiku 194 of 208), no wrong picks
Quoted vs unquoted YAML	4 runs × 3 models per version	Both used 12 of 12; strict readers refused the unquoted one
Packet scan	9 packets, 2 of them Anthropic's as controls	One real problem, in our own card
Review of a page that writes skills	15 mock tests	Live 2 of 15; patched 15 of 15

Read small numbers kindly. Most cards had 10 to 16 runs per model; in the template test, each card had 3. A difference of one or two is noise. The large gaps are not. We wrote the prompts; they were not collected from buyers.

2. What decides whether Claude picks a skill

Claude chooses from the description alone.

• Lead with the job and the moment. Our product-photo card says "use it whenever shoes, gear, bottles or any product are shot for Instagram". It was used on 36 of 36 of its runs and on no one else's. Two cards whose template said to use them "whenever the user names" the card scored 13 of 30 and 15 of 30; nobody asks for a tool by name. Rewritten to name the job, each scored 30 of 30.
• Name exact jobs, not a category. "Whenever a science question needs a number worked out" got our calculator card used 4 times in 36 on six such prompts; in the other 32, Claude worked it out alone. Naming each job, and why to run the script, made it 34 of 36. We changed both at once, so we cannot split the credit.
• Say "Not for…". Our running and cycling cards were picked 11 times for prompts not theirs; 10 were a diet plan or a pain to diagnose. A request to diagnose a numb wrist went to the cycling card on all 6 runs. Adding "Not for diagnosing pain, numbness or injury; that is a clinician's" cut wrong picks to 0. Hits held: 30 of 36 for running, 33 to 34 of 36 for cycling.
• A form check is not a trigger score. One line passed our whole checklist and told Claude it had "Scored 98 of 100". Claude used it 13 times in 30.
• The rules carry. Thirteen new cards, cut to these rules on 9 and 10 October, scored 585 of 624 on their first lines; eight of the thirteen scored 46 or more of 48 without a rewrite.
• One stray word can steal. Our tutor card's first line said it helps with "learning a new skill". On Haiku it took a prompt asking whether Claude would pick a skill, twice, and scored 8 of 16 on its own prompts. With the word gone and the neighbour named under "Not for", it scored 48 of 48. Our accessibility card's "brand palette" pulled in a request for a slide deck about brand colours, once; reworded, 48 of 48.

3. Findings for the platform

3.1 Strict YAML and what Claude Code accepts

What we saw. Claude Code loads a SKILL.md whose unquoted description holds a colon and a blank. Strict readers refuse it.

Evidence. One invented skill, in two versions differing only in quoting. Claude Code 2.1.296 used each 12 times in 12. PyYAML and the skill-creator's validator refused the unquoted one: "mapping values are not allowed here". Our own trigger-test card shipped this way. Our 12-point audit passed it, and it scored 30 of 30 on trigger runs, because our harness quotes every description it installs. The skill-creator's trigger loop, in our copy, rewrites the description as a block scalar. Both test a cleaned copy, not the author's file.

Why it matters. The author gets no signal. A strict upload or tool may refuse the file later, in front of a buyer.

A suggestion. One shared SKILL.md parser and validator across Claude Code, claude.ai, the API and the skill-creator. Until then, a warning in Claude Code when frontmatter would fail a strict read.

Not tested. We did not upload to claude.ai or use the Skills API. That they would refuse is our inference from the validator.

3.2 Built-in skills win plain requests, and authors cannot see it

What we saw. On plain document requests, Anthropic's skills were chosen over ours.

Evidence. Our PDF card was used 9 times in 36; the PDF skill took 23 of its 27 misses. It won 6 of 6 only on what it alone does: sign a PDF and seal it so a later change shows. A line saying it works on the buyer's machine with nothing uploaded, and naming its neighbours, raised it to 21 of 36. Our slides card was used 11 times in 36; PowerPoint took 22 of 25 misses. It won slides with a PNG of each slide 6 of 6.

Why it matters. If a buyer only wants two PDFs merged, the built-in doing it is fine. But we saw this only because our harness read Claude Code's event stream. The skill-creator's loop records only whether one skill fired, on one model. An author learns that a skill was skipped, not what took its place.

A suggestion. Author-side diagnostics in trigger tools: which skill was chosen instead, and on which model. In a marketplace, per-skill counts of offered and opened, with no user's prompt exposed.

Not tested. Our built-ins were stand-ins carrying the public descriptions. The Claude app's own may score differently.

3.3 Haiku answers on its own more often

What we saw. The smallest model most often answered without any skill.

Evidence. On our first ten cards' own prompts, before rewrites, the right card was used 62 times in 100 on Haiku, 79 on Sonnet and 78 on Opus. No skill at all: 27, 8 and 12. Under the old template, hits were 8, 13 and 18 of 27, every miss "no skill"; after the fix, 26, 27 and 25. The thirteen new cards showed the same shape on their first lines: Haiku 170 of 208, Sonnet 208 of 208, Opus 207 of 208. Five of the six rewrites that night were made to lift Haiku; the sixth fixed a Sonnet miss in a trigger gate. Conversely, Opus left near-misses alone least often (40 of 47, against 44 of 48 and 45 of 48) and made 5 of the 10 wrong picks on questions that belong to a clinician.

Why it matters. An author who tests on one model ships for all of them, and a skill's hand-off text only helps if it is loaded.

A suggestion. Trigger tools that run on every model a skill's users may have, by default, and short guidance on what smaller models need from a description.

Not tested. We ran by alias on one day. The models behind the aliases will change.

3.4 Refusals happen without opening the skill

What we saw. Claude declines harmful requests by itself, before it opens a skill.

Evidence. In a trigger gate for the science card, Claude declined a rocket-fuel request, a dosing request and a request to claim a cure, without opening the card. The card fired 0 of 3 times, so the gate failed at 4 of 7. That is good behaviour, so we changed our rule: such refusals are judged on the answer, and the gate passes at 4 of 4. Answer tests caught what trigger tests cannot: on a running injury, Haiku and Sonnet both pointed to a physio, as the card should, but wrote a full rehab plan first. That is the card's behaviour, and we have not changed it yet.

Why it matters. A skill's refusal text is often never read on the prompts it was written for. Only the answer shows whether it works.

A suggestion. Eval guidance that separates trigger cases from refuse cases and grades refusals on the answer.

Not tested. The gate ran once, on Sonnet; the answer tests once, on Haiku and Sonnet. Opus was not run.

3.5 Skills are a supply-chain surface

What we saw. The SKILL-INJECT preprint (arXiv 2602.20156) measured up to 80% attack success from injected skill files. Script payloads were stronger than text, and an injected description added roughly 11 to 14 points. It calls screening a partial defence and says simple input filtering will not solve the problem.

Evidence. Of 9 packets we scanned, the only blocking problem was ours: the unquoted description from 3.1. Four rules changed after false alarms on real packets, one a local Unix socket in Anthropic's Word skill. Our first scanner, placed inside a packet, failed its own scan: 3 blocking findings for direction-flipping characters written into its pattern lists, and 31 warnings for the strings it looks for. Two tampered copies of a card, one with hidden instructions and one with its watermark stripped, were both caught.

We also reviewed a page that writes a skill from a visitor's request. The live page passed 2 of 15 tests; the patch passes 15 of 15. Its second screen saw only the first 2,500 characters of SKILL.md, and two of five checks were hard-coded to pass. The tests used mocks, not a real model. The patch is now live on the page. Anyone can skip screens that run in a browser. Our fingerprint, a public SHA-256 of SKILL.md, proves the files match, not who made the card or that it was screened.

Why it matters. Tools that guard skills look like attacks to a pattern scanner, and a hash without a signer proves less than buyers may assume.

A suggestion. At install, show a content hash and, where one exists, a publisher's signature over a manifest of every file, and warn when a file has changed since signing. Scan in the marketplace and plugin install path, including scripts, hidden characters and frontmatter, and present a pass as "nothing known was found".

Not tested. We built no attacks and injected nothing into Claude. Our scanner reads patterns; a reworded attack will pass it.

4. What we hold ourselves to

• What we never make. Cards that diagnose, treat or set doses; help with weapons, explosives, malware or break-ins; impersonate anyone; produce sexual content or involve minors in anything sexual; harass, scam, give gambling tips, persuade on politics, collect private data, or touch self-harm or illegal drugs. A card may name these only to refuse them.
• Hand-offs. Every card must say what it is not for and point to a doctor, nurse or pharmacist, a legal adviser, a regulated financial adviser, or emergency services.
• Four checks before sale. A house audit, a trigger test, a packet scan, and a SKILL.md fingerprint buyers can check. The thirteen new cards passed all four. We have not yet confirmed all four on every older card on sale, and will say which have.
• A second read for sensitive cards is our written rule, but who does it is not yet confirmed, so we do not claim it.
• No claim until real people prove it. Our first study needs ten makers and has no result yet. All ten will be reported, bad results included; the claim may be weakened, never strengthened.
• Monthly re-tests are planned, because the models behind the aliases change.

5. Limits of this report

• One studio, one day, our own cards and prompts.
• Claude Code only. The Claude app, claude.ai uploads and the Skills API were not tested.
• Small samples. Trigger runs stopped at the first skill, so they say nothing about what a skill did once open.
• Two of the first ten cards were tested on reconstructed descriptions.
• The safety work is scans, code review and mocks, not attack tests.
• We sell skills and gain when they are picked. We have tried to weigh failures and successes alike.

6. Sources

• Trigger tests: THE-TRIGGER-TEST-KIT/ (START-HERE.md, CARD-DESCRIPTION-REGISTER.md, TRIGGER-PROMPTS-80.json, TEN-CARDS-AS-SHIPPED.json, RESULTS-AS-SHIPPED.jsonl, RESULTS-V2-PROPOSED.jsonl, SCORES-AS-SHIPPED.md/.json, SCORES-V2-PROPOSED.md/.json, triggertest.py); seating/score-seating-v1.md, -v2.md; obstest/score-obs.md, score-obs-v2.md; packetscantest/, marstest/, lasertest/, tutortest/, computetest/, experimenttest/, a11ytest/, carbontest/, datasettest/, productiontest/, linkbudgettest/, unitstest/, airisktest/ (each card's prompts, results, scores and seat record).
• The guide: pages/WRITING-THE-LINE-FREE-GUIDE-2026-10-09.md and WRITING-THE-LINE-NUMBERS-2026-10-09.md.
• YAML: packetscantest/YAML-EXPERIMENT.md and yamlexp-evidence/.
• Safety: scanner/README.md, scanner/SCAN-RESULTS.md, packetscantest/SCANNER-COPY-CHANGES-2026-10-09.diff, redteam/THE-MINT-REVIEW-2026-10-09.md.
• Evals and rules: evals/RERUN.md, pages/HOUSE-RULES-2026-10-09.md, flagship/README.md, flagship/STUDY-PROTOCOL.md.
• The skill-creator skill as installed here: SKILL.md, scripts/run_eval.py, scripts/quick_validate.py.
• "Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks." arXiv 2602.20156, February 2026.