mirror of
https://github.com/msitarzewski/agency-agents.git
synced 2026-09-04 09:10:50 +03:00
docs(contributing): tell contributors about the output eval, and why it's there (#830)
Adds the regression eval landed in #829 to the two places contributors actually read: item 8 of the tool-integration checklist (beside #772's install-suite item 7) and a new item 7 under "Before Submitting", the path every agent contributor takes. Adding or editing an agent changes the generated product and so flips the drift manifest; the text says so plainly, frames --update as the expected next step rather than a failure, and asks for the refreshed manifest to be committed alongside the change. Closes with a short note on why the checks exist: people are building remarkable things on these agents and thousands rely on them daily, so a slip in one converter reaches all of them at once — running the suite is how we keep that smooth for everyone downstream. Refs #829 #828 #772 Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5.1
parent
04eadbd3e5
commit
5457373419
@@ -245,6 +245,15 @@ Want agency-agents to install into a new tool (a CLI, editor, or agent runtime)?
|
||||
contract: where files land, that `--path` beats the tool's env var, that
|
||||
`--division` / `--agent` / `--agents-file` filter, that `--dry-run` writes
|
||||
nothing, and that paths with spaces survive. CI runs it on Linux and macOS.
|
||||
8. **Run `./scripts/test-convert-outputs.sh`** — it must pass. It regenerates
|
||||
every tool's output into a scratch directory and checks the *product*, not
|
||||
the syntax: every agent's description round-trips intact, every generated
|
||||
file parses with a real YAML/TOML parser, every tool emits exactly one output
|
||||
per agent, and every source file parses the way the desktop app reads it.
|
||||
When you've changed a converter on purpose it will report **manifest drift**
|
||||
— that's expected. Look over what changed, run it again with `--update`, and
|
||||
commit the refreshed `scripts/convert-outputs.sha256` so reviewers can see
|
||||
the blast radius at a glance. CI runs it on every PR.
|
||||
|
||||
If your PR commits the converted output (the generated `integrations/<tool>/*` files), CI and review will ask you to remove it and add the `.gitignore` rule instead.
|
||||
|
||||
@@ -301,6 +310,9 @@ We love ambitious ideas — a [Discussion](https://github.com/msitarzewski/agenc
|
||||
4. **Define Metrics**: Include specific, measurable success criteria
|
||||
5. **Proofread**: Check for typos, formatting issues, clarity
|
||||
6. **Check it's original**: Run `./scripts/check-agent-originality.sh path/to/your-agent.md`. It compares your agent against the whole roster and flags near-duplicates (a swapped country/platform name won't fool it). A new agent should be genuinely new — if you're localizing for a market, make the platforms, tactics, and examples actually different, not a find-replace.
|
||||
7. **Check it comes through every tool intact**: Run `./scripts/test-convert-outputs.sh`. It regenerates every tool's output and confirms your agent survives each converter — description round-tripped, files parsing, nothing dropped — and that its frontmatter parses the way the desktop app reads it. Adding or editing an agent changes the generated product, so it will report **manifest drift**; that's expected, not a failure on your part. Look over the change, run it again with `--update`, and commit the refreshed `scripts/convert-outputs.sha256` alongside your agent. CI runs the same check.
|
||||
|
||||
A word on why these checks exist. People are building genuinely remarkable things on top of these agents, and thousands rely on them every day across a dozen different tools. That's wonderful — and it means a small slip in one converter, or a stray quote in one file, quietly reaches all of them at once. Running the suite locally is how we keep that smooth for everyone downstream. It takes about a minute, and it means your work arrives exactly as you wrote it, in every tool, for everyone. Thank you for taking the extra step — it's a real kindness to people you'll never meet.
|
||||
|
||||
### Submitting Your PR
|
||||
|
||||
|
||||
Reference in New Issue
Block a user