creator/agency-agents
1
0
Fork 0
mirror of https://github.com/msitarzewski/agency-agents.git synced 2026-07-19 06:19:40 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat: add agent originality check (script + CI + docs) (#560)

Browse Source 
Adds scripts/check-agent-originality.sh, which flags new agents that
substantially duplicate an existing one. It compares each candidate
against the whole roster (and other files in the same change set) using
entity-neutralized 8-word shingle overlap, so a find-replace "re-skin"
that only swaps a country/platform name can't slip past review.

- CI: new "Check agent originality" step in lint-agents.yml runs it on
  changed agent files; a >=40% match fails the build.
- Docs: CONTRIBUTING.md gains a self-run "before submitting" step, a
  checklist item, and a "things we'll always close" bullet for re-skins.

Calibration: across the existing 184-agent library the worst same-pair
similarity is ~1.5% (median 0%), so the WARN >=20% / FAIL >=40% defaults
leave a wide margin against false positives.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Michael Sitarzewski
2026-06-03 12:08:28 -05:00
committed by GitHub
parent 083ce47e13
commit 5032f7e75c
3 changed files with 187 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CONTRIBUTING.md
+3
View File
@@ -264,6 +264,7 @@ We love ambitious ideas — a [Discussion](https://github.com/msitarzewski/agenc
#### Things we'll always close
- **Committed build output**: Generated files (`_site/`, compiled assets, converted agent files) should never be checked in. Users run `convert.sh` locally; all output is gitignored.
- **PRs that bulk-modify existing agents** without a prior discussion — even well-intentioned reformatting can create merge conflicts for other contributors.
- **Near-duplicate "re-skins"**: New agents that are find-replace copies of an existing one (e.g. swapping a country or platform name) rather than genuinely new specialists. Run `scripts/check-agent-originality.sh` before submitting — CI runs it automatically.
### Before Submitting
@@ -272,6 +273,7 @@ We love ambitious ideas — a [Discussion](https://github.com/msitarzewski/agenc
3. **Add Examples**: Include at least 2-3 code/template examples
4. **Define Metrics**: Include specific, measurable success criteria
5. **Proofread**: Check for typos, formatting issues, clarity
6. **Check it's original**: Run `./scripts/check-agent-originality.sh path/to/your-agent.md`. It compares your agent against the whole roster and flags near-duplicates (a swapped country/platform name won't fool it). A new agent should be genuinely new — if you're localizing for a market, make the platforms, tactics, and examples actually different, not a find-replace.
### Submitting Your PR
@@ -308,6 +310,7 @@ We love ambitious ideas — a [Discussion](https://github.com/msitarzewski/agenc
[How have you tested this agent? Real-world use cases?]
## Checklist
- [ ] Original — not a near-duplicate (ran `scripts/check-agent-originality.sh`)
- [ ] Follows agent template structure
- [ ] Includes personality and voice
- [ ] Has concrete code/template examples