feat: adding new skills, including testing patterns and methodologies, along with bundled resources for better usability.

2026-07-30 08:26:54 +03:00 · 2026-03-30 12:18:00 +07:00
parent 0ff5ae4082
commit 7fa9a48c6c
89 changed files with 25808 additions and 923 deletions
@@ -1,9 +1,11 @@
+---
+name: testing-anti-patterns
+description: >
+  Trigger this skill whenever writing, reviewing, or debugging tests. Activate for keywords like "mock", "stub", "test helper", "flaky test", "test passes but bug ships", "false positive", "test coverage", or when tests seem unreliable. Also trigger when reviewing test code in PRs, when tests pass but production breaks, when someone proposes heavy mocking, or when test failures are intermittent. If any test smells wrong or feels like it is not actually verifying real behavior, this skill applies.
+---
+
 # Testing Anti-Patterns

-## Description
-
-Common testing mistakes that create false confidence in test suites. Learn to recognize and avoid these patterns that make tests pass while failing to verify actual behavior.
-
 ## When to Use

 - Writing new tests
@@ -11,6 +13,12 @@ Common testing mistakes that create false confidence in test suites. Learn to re
 - Debugging flaky or unreliable tests
 - When tests pass but bugs still ship

+## When NOT to Use
+
+- Writing production code that is not test-related
+- Test framework configuration or setup (e.g., jest.config, vitest.config)
+- Performance testing or load testing scenarios
+
 ---

 ## The Five Anti-Patterns
@@ -256,3 +264,9 @@ Mocks should never:
 - Create false confidence

 ---
+
+## Related Skills
+
+- `methodology/test-driven-development` -- TDD naturally prevents most testing anti-patterns by requiring tests before implementation
+- `testing/pytest` -- Python-specific testing best practices that complement anti-pattern awareness
+- `testing/vitest` -- TypeScript/JavaScript-specific testing best practices that complement anti-pattern awareness
@@ -0,0 +1,183 @@
+# Testing Anti-Pattern Catalog
+
+Quick reference of common testing anti-patterns. Each entry includes: what it looks like, why it's a problem, and how to fix it.
+
+---
+
+## 1. The Ice Cream Cone
+
+**Symptom:** Most tests are E2E or integration tests. Few or no unit tests. The test pyramid is inverted.
+
+**Root cause:** Tests were written after the feature, following user flows instead of testing isolated logic. Or the code is tightly coupled, making unit tests hard to write.
+
+**Impact:** Test suite is slow, brittle, and expensive to maintain. Failures are hard to diagnose because tests cover too much at once.
+
+**Fix:** Refactor toward the test pyramid. Extract business logic into pure functions and unit test them. Reserve E2E tests for critical user flows only. Target ratio: 70% unit, 20% integration, 10% E2E.
+
+---
+
+## 2. The Mockery
+
+**Symptom:** Tests mock so aggressively that they're testing the mocks, not the actual code. The thing under test has all its dependencies replaced.
+
+**Root cause:** Code has too many dependencies, or the developer equates "isolated" with "mock everything."
+
+**Impact:** Tests pass even when the real code is broken. Refactoring breaks every test because mocks are coupled to implementation details.
+
+**Fix:** Only mock external boundaries (network, database, filesystem, clock). Use real implementations for internal collaborators. If you need too many mocks, the code has too many dependencies — refactor first.
+
+---
+
+## 3. The Slow Suite
+
+**Symptom:** Test suite takes more than a few minutes to run. Developers skip tests locally and only run them in CI.
+
+**Root cause:** Too many integration/E2E tests, tests that hit real databases or network, no test parallelization, expensive setup/teardown.
+
+**Impact:** Developers stop running tests, bugs slip through, CI becomes a bottleneck.
+
+**Fix:**
+- Profile the suite to find the slowest tests
+- Replace slow integration tests with fast unit tests where possible
+- Use in-memory databases for integration tests
+- Parallelize test execution
+- Target: unit suite under 30 seconds, full suite under 5 minutes
+
+---
+
+## 4. The Flaky Test
+
+**Symptom:** Test passes most of the time but fails unpredictably. Re-running usually makes it pass.
+
+**Root cause:** Race conditions, time-dependent logic, shared mutable state between tests, reliance on external services, non-deterministic ordering.
+
+**Impact:** Team loses trust in tests. "Oh that one's flaky" becomes an excuse to ignore real failures. CI results become meaningless.
+
+**Fix:**
+- Isolate the flaky test and run it 100 times to confirm flakiness
+- Check for: shared state, date/time usage, async timing, test ordering
+- Fix the root cause (don't just add retries)
+- Quarantine truly unfixable flaky tests while investigating
+
+---
+
+## 5. The Assertion-Free Test
+
+**Symptom:** Test runs code but doesn't assert anything meaningful. It only checks that no exception was thrown.
+
+```python
+# Bad — this tests nothing useful
+def test_process_data():
+    process_data(sample_input)  # No assertion
+```
+
+**Root cause:** Test was written to hit a coverage target rather than verify behavior.
+
+**Impact:** False sense of security. Code "has tests" but bugs go undetected.
+
+**Fix:** Every test must assert on the outcome. Ask: "What behavior am I verifying?" If you can't answer, the test isn't testing anything.
+
+```python
+# Good — asserts the actual behavior
+def test_process_data_calculates_total():
+    result = process_data(sample_input)
+    assert result.total == 42.0
+```
+
+---
+
+## 6. The Copy-Paste Test
+
+**Symptom:** Test file has blocks of nearly identical code repeated with minor variations. Tests are long and look like each other.
+
+**Root cause:** Developer tested a new case by copying an existing test and tweaking values instead of extracting a pattern.
+
+**Impact:** Maintenance nightmare. A change to the interface requires updating dozens of near-identical tests. Easy to introduce subtle bugs in copies.
+
+**Fix:** Use parameterized tests for variations on the same behavior:
+
+```python
+# Python — pytest.mark.parametrize
+@pytest.mark.parametrize("input,expected", [
+    ("hello", "HELLO"),
+    ("", ""),
+    ("123", "123"),
+])
+def test_to_upper(input, expected):
+    assert to_upper(input) == expected
+```
+
+```typescript
+// TypeScript — test.each (vitest/jest)
+test.each([
+  ["hello", "HELLO"],
+  ["", ""],
+  ["123", "123"],
+])("to_upper(%s) returns %s", (input, expected) => {
+  expect(toUpper(input)).toBe(expected);
+});
+```
+
+---
+
+## 7. The Time Bomb
+
+**Symptom:** Test passes today but will fail on a future date, or fails on certain days/times (new year, month boundary, DST change, leap year).
+
+**Root cause:** Test uses `Date.now()`, `new Date()`, or similar without controlling the clock. Assertions are hardcoded to specific dates.
+
+**Impact:** Sudden failures on specific dates. CI breaks on January 1, or during DST transitions.
+
+**Fix:** Always inject or mock the clock:
+
+```python
+# Python — freeze time
+from freezegun import freeze_time
+
+@freeze_time("2025-06-15T12:00:00Z")
+def test_expiry_check():
+    assert is_expired(created_at="2025-06-14T12:00:00Z", ttl_hours=23)
+```
+
+```typescript
+// TypeScript — vitest fake timers
+vi.useFakeTimers();
+vi.setSystemTime(new Date("2025-06-15T12:00:00Z"));
+expect(isExpired(createdAt, 23)).toBe(true);
+vi.useRealTimers();
+```
+
+---
+
+## 8. The Hidden Dependency
+
+**Symptom:** Test passes locally but fails in CI, or fails when run in isolation but passes as part of the full suite.
+
+**Root cause:** Test depends on external state that isn't set up by the test itself: a running database, a file on disk, an environment variable, output from a previous test, or global state modified by another test.
+
+**Impact:** Tests are order-dependent, environment-dependent, and unreliable. Debugging failures requires understanding the entire test suite's execution order.
+
+**Fix:**
+- Each test must set up and tear down its own state
+- Use fixtures (pytest fixtures, beforeEach/afterEach) for shared setup
+- Run tests in random order to catch hidden dependencies
+  ```bash
+  pytest -p randomly    # Python
+  vitest --sequence.shuffle  # vitest
+  ```
+- Never rely on test execution order
+
+---
+
+## Quick Decision Table
+
+| Symptom | Likely Anti-Pattern | First Action |
+|---------|-------------------|--------------|
+| Tests are slow | Ice Cream Cone or Slow Suite | Profile, find the slowest tests |
+| Tests break on refactor | The Mockery | Reduce mocks, test behavior not implementation |
+| Tests fail randomly | Flaky Test | Isolate and run 100x |
+| High coverage but bugs slip through | Assertion-Free Test | Audit assertions in coverage-targeted tests |
+| Tests are hard to maintain | Copy-Paste Test | Extract parameterized tests |
+| Tests fail on certain dates | Time Bomb | Inject/mock the clock |
+| Tests fail in CI only | Hidden Dependency | Run locally in random order |
+| Tests pass but code is clearly broken | The Mockery or Assertion-Free | Check what's actually being asserted |