The description is the only signal an agent sees at discovery time, so
overlapping descriptions cause misrouting. Nothing in CI checked for that.
- tools/lint-descriptions.py enforces name==folder, description <=1024
chars, terminal punctuation (a truncation canary), a trigger clause, a
negative trigger, and a 500-line body cap. Pre-existing failures are
grandfathered in tools/lint-baseline.json so this blocks new debt only;
the baseline may shrink and never grow.
- tools/detect-collisions.py scores every description pair by TF-IDF
cosine and ratchets the count of unreviewed near-duplicates. It strips
negative-trigger clauses before vectorizing: those name the sibling
skill on purpose, so scoring them would make correct disambiguation
raise a pair's similarity.
- wire both into validate-skills.yml, along with agentskills conformance,
an index.json freshness check, and a guard that fails the build if a
regex frontmatter parser is reintroduced.
- broaden the path filters from tools/validate-skill.py to tools/**, as
noted when #105 merged.
All five gates verified to fail on deliberately broken input.
index.json shipped 604 of 817 descriptions truncated to their first line.
The cause was the inline regex parser in update-index.yml: it reconstructed
multi-line descriptions only for the YAML block-scalar styles ('>' and '|')
and silently dropped continuation lines for every other style.
A census of the corpus explains the blast radius:
block scalar (description: >-) 43
single-quoted multiline 278
plain unquoted multiline 496
single-line 0
So 774 of 817 skills (94.7%) used a style the parser mishandled. Commit
d56fc0a7 had fixed only the 43 block-scalar files, and CONTRIBUTING.md
recommends that one working style, which is why it stayed hidden.
- add tools/skill_frontmatter.py, the single PyYAML-backed loader
- add tools/generate-index.py so generation is testable outside CI, with
a --check mode for use as a gate
- delete the hand-rolled parsers from validate-skill.py (98 lines) and
validate-agentskills.py, routing both through the shared loader
- implement the reserved-word check that agentskills-skill.schema.json
names validate-agentskills.py as the enforcement point for
Verified by a differential harness against yaml.safe_load ground truth:
index-vs-source mismatches 606 -> 0.
The standard requires only name+description; additional top-level keys are
permitted metadata. Fixes:
- validator no longer counts extra top-level keys as violations (reports them
as informational only).
- angle-bracket check now ignores YAML block-scalar indicators (`>`, `>-`,
`|`), which were false-positiving on multi-line descriptions; no skill has
genuine angle-bracket content.
- schema additionalProperties false -> true to match.
Audit result: 817/817 compliant.
- tools/agentskills-skill.schema.json: strict JSON Schema for the
agentskills.io SKILL.md frontmatter standard (name+description required;
optional license/compatibility/metadata/allowed-tools; no other top-level
keys).
- tools/validate-agentskills.py: read-only compliance validator (also checks
name==directory and the no-angle-brackets rule).
- Add hardware-firmware-security as a canonical subdomain (folding in the
firmware-analysis/firmware-security aliases). The 4 new hardware/firmware
skills failed validation because the subdomain was not in the allowed set.
- Skip skills/*.bak/ backup directories in --all mode; they have no SKILL.md
and were producing 21 false failures.
- Result: validate-skill.py --all now reports 817/817 passing, exit 0.
Issues found in review:
1. tools/validate-skill.py: parse_frontmatter operated on the stripped line, so
an indented nested `name:` (under framework-mapping lists, e.g.
`name: 'Create Fake Materials: Fake Website'`) clobbered the skill's
top-level `name`. That produced 94 spurious "invalid kebab-case name"
failures out of 762. Now indented (non-list) key lines are ignored, so only
top-level keys define frontmatter fields. Result: 762/762 pass.
2. Two divergent validators: the CI workflow had its own weaker inline parser
(no subdomain/tag/description checks) requiring a different field set than
tools/validate-skill.py. CI now delegates to tools/validate-skill.py --all
(single source of truth); REQUIRED_FIELDS aligned to include
version/author/license. The duplicate-name and stats steps are unchanged.
3. README: added an explicit authorized-&-lawful-use disclaimer next to the
existing "not affiliated with Anthropic" note, since the library ships
offensive/dual-use techniques.
No skill content changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three issues fixed:
1. Description list check — added elif isinstance(desc, list) branch that
emits 'Description must be a string value, not a list'. Previously the
block was silently skipped when YAML returned a list, causing the skill
to pass without validating the description field.
2. tools/README.md synced — updated description constraint from '20-500
characters' to 'at least 50 characters (no upper limit)' to match the
current code (DESCRIPTION_MIN_CHARS=50, no max enforced).
3. --all with wrong CWD now exits 1 — if glob returns no skill dirs,
the script prints an error and exits with code 1 instead of reporting
'Total: 0 Passed: 0 Failed: 0' and exiting 0, which would cause CI to
silently pass while validating nothing.
All 754 skills continue to pass (0 regressions).
Required changes:
- Error handling: IOError and UnicodeDecodeError already wrapped in
try/except from previous commit — still present and correct.
- ALLOWED_SUBDOMAINS: synced with actual repo usage (audited all 754
skills). identity-access-management (34 skills) added; identity-security
was the placeholder in its place.
New in this commit:
1. Description minimum: raised from 20 → 50 chars to align with other
repo tooling as requested.
2. Folded scalar support: parse_frontmatter now handles YAML `>-` and `>`
folded scalars, preventing incorrect parse of multi-line descriptions.
Added a comment documenting the one remaining edge case (value-less key
followed by non-list content — treated as no-value, acceptable for
well-formed SKILL.md files).
3. Canonical subdomain warnings: alias subdomain values (e.g.
security-operations vs soc-operations) now print a WARN line pointing
to the canonical form, but are non-blocking. A _SUBDOMAIN_ALIASES dict
documents canonical/alias pairs explicitly.
4. Description upper limit: removed hard cap — folded scalars legitimately
produce long strings in existing skills.
5. PR description: removed false mention of type hints (there are none
in this file).
Validator now passes 754/754 skills in the repo with 0 errors.
- Wrap open() call in try/except for IOError and UnicodeDecodeError
to report clean errors instead of crashing on encoding issues
- Add all subdomains actually used by existing skills in the repo:
identity-access-management (33 skills), security-operations (28),
identity-and-access-management, zero-trust, ot-security, purple-team,
red-team, ai-security, social-engineering-defense, and others
- Remove identity-security as the canonical form is identity-access-management
Mahipal
Homan AnsariandClaude Opus 4.8
Julio César Suástegui