Adds scripts/test-convert-outputs.sh, the output half of the regression eval
(the install half landed in #828), and wires it plus the previously
un-wired test-agent-selection.sh (#779) into CI.
Why an eval at all: every converter bug so far passed lint and the existing
tests while the installed product was broken. #778 shipped a double-wrapped
description that was valid YAML, so a wrapper check passed; #817 dropped a
whole tool from --parallel and every remaining tool looked fine. Those are
invariant violations, not syntax errors.
Layer A (no history needed), for every agent x every converted tool:
round-trip parsed(generated).description == source description
strict-parse every generated frontmatter / TOML / YAML parses with a real
parser (kimi/vibe carry only an identifier: id == slug and the
prose file exists; aider/windsurf: "## Name" + description line)
count every tool emits exactly one output per roster agent
source every SOURCE frontmatter strict-parses and carries no leaked
quote — the desktop app reads sources with js-yaml (#473)
Layer B: scripts/convert-outputs.sha256, one aggregate hash per tool plus
divisions.json / tools.json / runbooks.json. A flipped line means outputs or a
contract changed; --update regenerates deliberately so review sees the blast
radius. Date-stable (no generated file embeds a date).
The expected side is derived by an INDEPENDENT strict parse of each source,
never by lib.sh's get_field: the generator uses get_field, so an expected
value derived the same way would move with a get_field bug and hide it —
which is exactly how #778 stayed invisible. That independence found two
shipping defects on the first green run:
- get_field returned only the first line of a multi-line plain scalar.
Three healthcare agents write their description as an indented
continuation; every generated output for them shipped it truncated
mid-sentence while the app showed the whole thing. get_field now folds
continuation lines the way YAML does (newline -> single space).
- get_field stripped only "field: " (one space). The same three files use
column-aligned frontmatter (name: X), so their generated names
carried leading whitespace in every tool's output. Plain-scalar padding
is now trimmed.
After both fixes get_field agrees with PyYAML on name and description for
all 273 sources. Acceptance: re-introducing #778's double-wrap, a dropped
tool, a divisions.json change, and an unquoted source each fail the eval
(the double-wrap via Layer A round-trip, not only the manifest).
Refs #778#817#473#810#826#828#779
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Reapply "test(install): add a regression suite for install.sh + CI on Linux and macOS (#772)" (#827)
This reverts commit 4bab3cf4a2.
* fix(install): refuse --path only for tools that would overwrite each other; re-land installer test suite (#772)
Re-lands the install.sh regression suite from #772 (reverted in #827) together
with the guard change that makes it pass, so CI goes green in one step.
Background. #825 made --path refuse more than one --tool, on #819's report that
several tools sharing one destination clobber each other. That was right in
spirit and over-broad in practice, and it was implemented without verifying
the premise. Measured by installing one agent with every tool into a sandbox
and comparing what landed:
<division>-<slug>.md (raw copy) claude-code, copilot
<slug>.md (converted) gemini-cli, opencode, qwen, zcode
agency-<slug>/SKILL.md antigravity, osaurus
Tools in the same group write identical filenames and silently overwrite each
other (qwen + gemini-cli lose a file while both print [OK]). Tools in different
groups coexist (claude-code + codex, claude-code + qwen). Every other tool's
output is distinct.
The guard now refuses --path only for a colliding pair, naming both tools and
the reason, and allows the rest. path_collision_group() holds the measured
table; re-measure if a converter's naming changes.
The suite's two-tool --path cases used claude-code + copilot, which collide:
the "installs exactly one agent" count of 1 was passing because copilot had
overwritten claude-code's identical file. They now use claude-code + codex and
assert that BOTH outputs survive, which a single count cannot show; a new case
asserts the colliding pair is refused. codex has no committed output, so those
cases convert (only the two raw-copiers work under --no-convert in a fresh
checkout).
Verified under bash 3.2 (the macOS CI leg): 29 passed, 0 failed, 1 xfail.
Guard spot-checked against all three measured groups plus a cross-group pair.
Suite, workflow and CONTRIBUTING note by @SergiorCode (#772). Refs #772#819#825#827.
Co-Authored-By: SergiorCode <SergiorCode@users.noreply.github.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(install): auto-convert never fired in a fresh checkout — README.md read as generated output
ensure_converted decided a converted tool's output was present if its
integrations/<tool>/ directory contained any file. Every one of those
directories ships a committed README.md, so in a fresh checkout the check
always found a file, skipped convert.sh, and the installer then hard-failed
"integrations/<tool> missing. Run convert.sh first." — the exact flow #426's
auto-convert was added to prevent. This affected every converted tool; it was
masked locally by generated outputs left behind in the working tree, and
surfaced only when the test suite ran in a clean checkout.
Only files other than README.md now count as generated output. Verified in a
clean worktree under bash 3.2: the suite's serial-control case (claude-code +
codex into one --path) now auto-converts codex and both outputs land;
29 passed, 0 failed, 1 xfail.
Refs #426
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: SergiorCode <SergiorCode@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* test(install): add a regression suite for install.sh + CI on Linux and macOS
install.sh is the largest script in the repo and has no tests. Every install
bug so far has been a silent one — agents copied to the wrong directory, a
path with a space split in two, a filter that installed everything — and the
only signal was a user noticing later.
scripts/test-install.sh pins the installer's observable contract:
* destinations: default $HOME/.claude/agents, --path override, tool env var
override, and --path winning over the env var
* selection: --division, --agent, --agents-file (comments/blank lines)
* --dry-run writes nothing; unknown --tool exits non-zero
* --link produces symlinks; a second run installs the same set, not dupes
* a destination containing spaces stays one directory
Expected counts are derived from divisions.json + lib.sh at runtime, so the
suite doesn't need updating when agents are added. Every case runs with HOME
pointed at a throwaway sandbox, so a broken default path can never write into
the real config. bash 3.2 + BSD userland, no new dependencies.
Verified it fails on the regressions it claims to catch: unquoting install_file
fails only the spaces case, neutering slug_allowed fails the four selection
cases, un-short-circuiting --dry-run fails the dry-run case, and ignoring the
env var in resolve_dest fails the env-override case.
CI runs it on ubuntu-latest and macos-latest (macOS ships bash 3.2, Linux
ships bash 5) plus bash -n over every script in scripts/.
* test(install): pin the parallel worker argument regression (#755)
Review feedback: the existing "paths with spaces" case selects a single tool,
so it stays on the serial path and never reaches the worker spawn where #755's
bug lives. Adds a case that does.
--tool claude-code,copilot --parallel --jobs 1 --agents-file <spaced path>
--path "<home>/My [Agents]/dest dir"
with a serial control immediately before it (same two tools, same spaced and
globbed --path, no --parallel) so a failure is attributable to the worker
hand-off rather than to the selection filter.
Marked xfail rather than a hard assertion: it fails on main today and passes
with #755 applied, and encoding a known-broken case as a hard failure would
turn CI red for reasons unrelated to whatever PR is being reviewed. xfail
never fails the suite; when the case starts passing it prints a note to
promote it to assert_eq (one-word edit). Measured on macOS bash 3.2.57:
main -> 25 passed / 1 xfail, #755 applied -> 26 passed / 0 failed, both
deterministic over repeated runs.
Note on --jobs 1: workers are still spawned through the same xargs/sh
hand-off, so argument propagation is exercised in full. Serializing them
keeps a second, unrelated defect out of this case — with two workers running
concurrently against one shared --path, the parent exits non-zero on ~3 runs
in 5 once the workers actually copy anything (one worker's cp fails with
ENOENT on the shared destination). That race is invisible on main only
because the workers currently install nothing at all; --jobs 1 or per-tool
destinations are clean. Reported in the PR discussion.
* Add Research division with Research Synthesist agent
New division for literature review, source evaluation, and evidence
synthesis. Wired into divisions.json, convert.sh, lint-agents.sh,
install.sh, and the lint-agents CI workflow.
* chore(hermes): regenerate agent count 272 -> 273 for research division (#770)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Prashant Raj Bista <prashant.bista.18@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
The app can't reliably resolve runbook rosters from display names (catalog
slugs are inconsistently division-prefixed, and names drift). This adds a
machine-readable manifest so the app reads rosters as data and maps each
slug to a catalog agent for one-click team deploy.
- strategy/runbooks.json: the 4 NEXUS scenarios (startup-mvp,
enterprise-feature, marketing-campaign, incident-response), each with
mode, duration, summary, doc, and a grouped roster. Every agents[] entry
is a verified slug = the agent .md filename stem (the corpus id), resolved
against the live roster — not a slugified display name. (Notably
"Senior Project Manager" is project-manager-senior, NOT
project-management-senior-project-manager, which naive mapping assumes.)
- scripts/check-runbooks.sh + .github/workflows/check-runbooks.yml: guard
(mirrors check-divisions.sh) failing the build if any roster slug doesn't
resolve to a real agent file, a doc path is missing, or JSON is malformed —
so renaming/removing an agent can't silently break the app's deploy.
All 64 slug references verified; guard passes and fails correctly.
Claude-Session: https://claude.ai/code/session_01WKnDRWM4izsB8WAXKszhsq
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Agents developed by Snark Health (github.com/snark-health).
Snark Health was founded by a practicing US physician with 25 years
of internal medicine and infectious disease experience and direct
leadership of a $2 billion risk-based Medicare bundled payment
contract with the US government, and a Kenyan engineer and operator
whose collaboration with the founding physician began in 1998 in
rural western Kenya. The frameworks in these files come from a team
that has delivered care in both US hospital systems and
resource-limited settings, managed actuarial risk under government
contract, and built health infrastructure across two continents
over 25 years.
AI Collective OS: snarkhealth.ai
Agent registry: snarkhealth.ai/registry
Mirrors check-divisions.yml. Runs scripts/check-tools.sh on every PR and on
push to main (no path filter) so any change to ALL_TOOLS in install.sh, the
converter set in convert.sh, or tools.json that breaks consistency fails the
build — same CI protection divisions.json already has.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
strategy/ holds 16 markdown files and ZERO have agent frontmatter — they're
playbooks (playbooks/phase-*.md), runbooks (runbooks/scenario-*.md), and briefs
(EXECUTIVE-BRIEF.md, QUICKSTART.md, nexus-strategy.md), not agent definitions.
There are 16 real agent divisions, 232 agents; strategy is not one of them.
#592 added `strategy` to lint-agents.sh AGENT_DIRS and the lint workflow paths
(to match divisions.json), which made CI lint those 16 frontmatter-less docs as
agents and fail every one with "missing frontmatter opening ---". So any PR
touching strategy/ broke CI. The original lint-agents.sh correctly excluded
strategy; #592 misread that deliberate exclusion as drift (same mistake as
integrations/ in #593).
Fix: remove strategy from convert.sh / lint-agents.sh AGENT_DIRS, the lint
workflow, and divisions.json; add it to NON_DIVISION_DIRS in check-divisions.sh.
divisions.json is now 16, matching the app's parse_agent count exactly.
Also add a content-derived backstop to check-divisions.sh: every division must
contain at least one .md with '---' frontmatter, or the build fails. This is
what stops a docs/playbook directory from being registered as an empty agent
division again — regardless of whether someone remembers the exclude list.
check-divisions.sh PASSES at 16; negative-tested that re-adding strategy fails
with "division 'strategy' has no agent files".
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#592 added `integrations` to AGENT_DIRS in convert.sh and lint-agents.sh and to
the lint workflow paths, to make those lists match divisions.json. That was
wrong: integrations/ is not a source-agent category — it's where convert.sh
WRITES per-tool conversions (e.g. openclaw output → integrations/openclaw/<agent>/SOUL.md).
It holds 957 conversion outputs across openclaw/opencode/qwen/antigravity, vs
248 real source agents in the 17 genuine categories.
Scanning integrations/ as source made the toolchain re-convert its own outputs:
the same agent appears under every tool (brand-guardian ×5), output slugs
collide, and convert.sh's last-writer-wins corrupts the catalog — which broke
downstream parity checks. convert.sh originally omitted integrations on purpose;
#592 misread that deliberate exclusion as drift.
Fix: drop integrations from convert.sh / lint-agents.sh AGENT_DIRS and the lint
workflow, remove it from divisions.json (it's not a division), and add it to
NON_DIVISION_DIRS in check-divisions.sh so the guard's canonical set is the real
17 source categories. The `strategy` additions from #592 were correct and stay.
check-divisions.sh now PASSES at 17 divisions consistent across divisions.json,
directories, scripts, and CI.
Note: integrations/mcp-memory holds 2 real source agents stranded in the output
tree; relocating them to a real category is left as separate follow-up.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add divisions.json — presentation metadata (label, icon, color) per division
Establishes a source of truth for how each division (top-level agent directory)
is presented: a display label, a Lucide icon name, and a brand color. Lets the
Agency Agents app (and any other tooling) render divisions consistently —
including fixing "GIS" (was title-cased to "Gis") and covering `gis` +
`integrations`, which had no metadata before.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Make divisions.json the source of truth + enforce in CI
divisions.json now drives the division set. Add scripts/check-divisions.sh
(CI: check-divisions.yml, runs on every PR with no path filter) which fails
if divisions.json disagrees with the directories on disk, the AGENT_DIRS
arrays in convert.sh / lint-agents.sh, or the lint-agents.yml path filters,
or if any entry lacks label/icon/color.
Fixes pre-existing drift surfaced by the new check: integrations was missing
from convert.sh and lint-agents.sh; integrations and strategy were missing
from lint-agents.sh and the lint workflow (so those agents weren't being
linted at all).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds scripts/check-agent-originality.sh, which flags new agents that
substantially duplicate an existing one. It compares each candidate
against the whole roster (and other files in the same change set) using
entity-neutralized 8-word shingle overlap, so a find-replace "re-skin"
that only swaps a country/platform name can't slip past review.
- CI: new "Check agent originality" step in lint-agents.yml runs it on
changed agent files; a >=40% match fails the build.
- Docs: CONTRIBUTING.md gains a self-run "before submitting" step, a
checklist item, and a "things we'll always close" bullet for re-skins.
Calibration: across the existing 184-agent library the worst same-pair
similarity is ~1.5% (median 0%), so the WARN >=20% / FAIL >=40% defaults
leave a wide margin against false positives.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds finance/ to AGENT_DIRS in all 3 scripts, CI workflow trigger paths, CONTRIBUTING.md category list, and README.md division roster. Also fixes duplicate sales entry in lint-agents.sh.
Expands CI lint workflow to trigger on academic/ changes. Hardens lint-agents.sh with file existence checks and portable word-count handling (macOS/BSD compatibility).
- Add sales/ to claude-code and copilot installer dir lists
- Add sales/ to lint-agents workflow trigger and changed-files filter
- Add LinkedIn Content Creator to marketing table in README
- Update agent count to 120
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add game-development to AGENT_DIRS in convert.sh, lint-agents.sh,
install.sh, and the CI workflow
- Remove maxdepth 1 from find commands so nested subdirectories
(e.g., game-development/unity/) are discovered
- Update CI git diff globs to **/*.md for recursive matching
- Remove strategy from lint AGENT_DIRS (playbooks, not agents)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prepares infrastructure for the upcoming Paid Media Division (PR #83).
Registers paid-media in convert.sh, lint-agents.sh, install.sh, and
the lint-agents GitHub Actions workflow.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix ((errors++)) causing early exit under set -e
- Fix command injection in workflow by using env variable
- Add strategy/ directory to lint paths and AGENT_DIRS
Adds a lint script and GitHub Actions workflow that checks agent
markdown files for required YAML frontmatter fields (name, description,
color) and recommends standard sections. Only changed files in PRs
are validated to avoid blocking on pre-existing issues.
Michael SitarzewskiandClaude Fable 5.1
Sergio Romero
hari
Pip, Agent PipandMichael Sitarzewski
Hank Selke
everforge
Yimikami
jiangnan