mirror of
https://github.com/msitarzewski/agency-agents.git
synced 2026-09-04 01:00:51 +03:00
Michael SitarzewskiandClaude Fable 5.1
04eadbd3e5
Adds scripts/test-convert-outputs.sh, the output half of the regression eval (the install half landed in #828), and wires it plus the previously un-wired test-agent-selection.sh (#779) into CI. Why an eval at all: every converter bug so far passed lint and the existing tests while the installed product was broken. #778 shipped a double-wrapped description that was valid YAML, so a wrapper check passed; #817 dropped a whole tool from --parallel and every remaining tool looked fine. Those are invariant violations, not syntax errors. Layer A (no history needed), for every agent x every converted tool: round-trip parsed(generated).description == source description strict-parse every generated frontmatter / TOML / YAML parses with a real parser (kimi/vibe carry only an identifier: id == slug and the prose file exists; aider/windsurf: "## Name" + description line) count every tool emits exactly one output per roster agent source every SOURCE frontmatter strict-parses and carries no leaked quote — the desktop app reads sources with js-yaml (#473) Layer B: scripts/convert-outputs.sha256, one aggregate hash per tool plus divisions.json / tools.json / runbooks.json. A flipped line means outputs or a contract changed; --update regenerates deliberately so review sees the blast radius. Date-stable (no generated file embeds a date). The expected side is derived by an INDEPENDENT strict parse of each source, never by lib.sh's get_field: the generator uses get_field, so an expected value derived the same way would move with a get_field bug and hide it — which is exactly how #778 stayed invisible. That independence found two shipping defects on the first green run: - get_field returned only the first line of a multi-line plain scalar. Three healthcare agents write their description as an indented continuation; every generated output for them shipped it truncated mid-sentence while the app showed the whole thing. get_field now folds continuation lines the way YAML does (newline -> single space). - get_field stripped only "field: " (one space). The same three files use column-aligned frontmatter (name: X), so their generated names carried leading whitespace in every tool's output. Plain-scalar padding is now trimmed. After both fixes get_field agrees with PyYAML on name and description for all 273 sources. Acceptance: re-introducing #778's double-wrap, a dropped tool, a divisions.json change, and an unquoted source each fail the eval (the double-wrap via Layer A round-trip, not only the manifest). Refs #778 #817 #473 #810 #826 #828 #779 Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
18 lines
1.2 KiB
Plaintext
18 lines
1.2 KiB
Plaintext
antigravity 3a6a9f27c2478c05093f8cecef99793870250cfc1774e4d8642d8d92001d4b0e
|
|
gemini-cli 389eaa64b91beef18e999471f227036aefcbf3ac8d87858cabcae39a594c5951
|
|
opencode 99af3bd8f155ea2ce9c5f9bfbf53ff1a083be1863cc998d2a40e63cd6314fff9
|
|
cursor 169d7c6e752490cfe8ad9a40a0e13a19f5bb641862d580bad9b23596a7d731f2
|
|
aider 9438f15e56aec67943b56deb21f459f4f4adeb457b96316309175c6ca6fcff30
|
|
windsurf e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
|
|
openclaw 3019e15be427bc9d38e22765dab57951fe7d91d352c1b04b61e045f8b360dd22
|
|
qwen 6602754c05df8a700dde9864afc3bf5275fa5c1b52a2c52771c63f0d5c4bf358
|
|
zcode 994ce45c33d46d2c694d201792fc1cd9dffcb1bcfe285ab46767d7d29f0831c6
|
|
kimi 0be25760acdcf1638c2605d05ffb80fe05f751b6b76e5b3ba67af82885118606
|
|
codex d666dfb2d78328d0cfccb6ebcdb3d35103e2280cca113cf0248a78bb66039150
|
|
osaurus 8c7b843ed0cfb8279ed75306c1744e14e49a1e6815a4ce7bb055d047b87a5cc6
|
|
hermes 8c124dbe0c8e73aed5481dcfa077eda553f3ff66e8ca5d28dacb4aba1d8dacdf
|
|
vibe 8c79416463420d73bfa7ec79f51df95568bb7bd73626e1107137f8fa5900c9ea
|
|
divisions.json a85d4ceeabe671051559e7703dfca074bf7a259ba26b61fb5e14ed36d54ae051
|
|
tools.json 2b9635406d8980dfde28849f965f2a22de384b4906128ec345ef6fc6246f127d
|
|
strategy/runbooks.json 2d0372782460694bddcbdb5ec04ff28ae6284a2c751888f8010e16033024ac49
|