diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index 7fdc6ae..fb7a7d0 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -176,7 +176,7 @@ Modes adjust communication style, output format, and problem-solving approach. |------|-------------|----------| | `default` | Balanced standard behavior | General tasks | | `brainstorm` | Creative exploration, questions | Design, ideation | -| `token-efficient` | Compressed, concise output | High-volume, cost savings | +| `writing-concisely` | Compressed, concise output | High-volume, cost savings | | `deep-research` | Thorough analysis, citations | Investigation, audits | | `implementation` | Code-focused, minimal prose | Executing plans | | `review` | Critical analysis, finding issues | Code review, QA | @@ -241,7 +241,7 @@ Control output verbosity for cost optimization. /mode token-efficient # Enable for entire session ``` -Reference: `.claude/skills/optimization/token-efficient/SKILL.md` +Reference: `.claude/skills/writing-concisely/SKILL.md` ## Context Management @@ -328,31 +328,32 @@ To use subagent mode: `/execute-plan [plan-file]` For strict TDD enforcement (no production code without failing test): - Use `/tdd [feature]` command -- Reference: `.claude/skills/methodology/test-driven-development/SKILL.md` +- Reference: `.claude/skills/test-driven-development/SKILL.md` ### Verification Requirements Enable mandatory verification before completion claims: -- Reference: `.claude/skills/methodology/verification-before-completion/SKILL.md` +- Reference: `.claude/skills/verification-before-completion/SKILL.md` ### Available Skills | Category | Skills | |----------|--------| -| **Languages** | python, typescript, javascript | -| **Frameworks** | fastapi, django, nextjs, react | -| **Databases** | postgresql, mongodb | -| **DevOps** | docker, github-actions | -| **Frontend** | tailwind, shadcn-ui | +| **Languages** | languages (Python, TypeScript, JavaScript) | +| **Backend** | backend-frameworks (FastAPI, Django, NestJS, Express) | +| **Frontend** | frontend (React, Next.js, shadcn/ui), frontend-styling (Tailwind, accessibility) | +| **Databases** | databases (PostgreSQL, MongoDB, Redis, migrations) | +| **DevOps** | devops (Docker, GitHub Actions, Cloudflare Workers) | | **Security** | owasp | | **API** | openapi | -| **Testing** | pytest, vitest | -| **Optimization** | token-efficient | -| **Developer Patterns** | error-handling, state-management, logging, caching, api-client, authentication | -| **Methodology - Planning** | brainstorming, writing-plans, executing-plans | +| **Testing** | testing (pytest, vitest, Jest), playwright | +| **Optimization** | writing-concisely | +| **Developer Patterns** | error-handling, state-management, logging, caching, api-client, authentication, background-jobs | +| **Methodology - Planning** | brainstorming, writing-plans, executing-plans, writing-skills | | **Methodology - Testing** | test-driven-development, verification-before-completion, testing-anti-patterns | | **Methodology - Debugging** | systematic-debugging, root-cause-tracing, defense-in-depth | -| **Methodology - Collaboration** | dispatching-parallel-agents, requesting-code-review, receiving-code-review, finishing-development-branch | +| **Methodology - Collaboration** | dispatching-parallel-agents, requesting-code-review, receiving-code-review, finishing-a-development-branch | +| **Methodology - Execution** | subagent-driven-development, using-git-worktrees, condition-based-waiting | | **Methodology - Reasoning** | sequential-thinking | Skills location: `.claude/skills/` @@ -367,7 +368,7 @@ Each skill includes: ### Sequential Thinking For complex problems requiring step-by-step analysis: -- Reference: `.claude/skills/methodology/sequential-thinking/SKILL.md` +- Reference: `.claude/skills/sequential-thinking/SKILL.md` - Activation: `/research --sequential [topic]` or use deep-research mode ## Environment Configuration @@ -455,9 +456,9 @@ pnpm install ## Kit Version - **Claude Kit Version**: 3.0.0 -- **Last Updated**: 2026-03-30 +- **Last Updated**: 2026-04-18 - **Compatible with**: Claude Code 1.0+ -- **Total Skills**: 38 (with YAML frontmatter, bundled resources) +- **Total Skills**: 36 (with YAML frontmatter, bundled resources) - **Total Commands**: 27+ - **Total Agents**: 20 - **Behavioral Modes**: 7 diff --git a/.claude/agents/brainstormer.md b/.claude/agents/brainstormer.md index 5ce1bda..2d694f2 100644 --- a/.claude/agents/brainstormer.md +++ b/.claude/agents/brainstormer.md @@ -277,7 +277,7 @@ Starting from fundamentals, what's the best way to solve this? For enhanced interactive brainstorming, use the superpowers methodology: -**Reference**: `.claude/skills/methodology/brainstorming/SKILL.md` +**Reference**: `.claude/skills/brainstorming/SKILL.md` Key principles from superpowers methodology: - **One question per message**: Ask single questions, wait for response diff --git a/.claude/agents/code-reviewer.md b/.claude/agents/code-reviewer.md index d1693b7..c108746 100644 --- a/.claude/agents/code-reviewer.md +++ b/.claude/agents/code-reviewer.md @@ -183,7 +183,7 @@ For enhanced code review workflows, use the superpowers methodology: ### Requesting Reviews -**Reference**: `.claude/skills/methodology/requesting-code-review/SKILL.md` +**Reference**: `.claude/skills/requesting-code-review/SKILL.md` Include in review requests: - Scope definition (files, lines changed) @@ -193,7 +193,7 @@ Include in review requests: ### Receiving Reviews -**Reference**: `.claude/skills/methodology/receiving-code-review/SKILL.md` +**Reference**: `.claude/skills/receiving-code-review/SKILL.md` Process feedback by category: - **Critical**: Must fix before proceeding @@ -204,7 +204,7 @@ Process feedback by category: When using subagent-driven development: -**Reference**: `.claude/skills/methodology/executing-plans/SKILL.md` +**Reference**: `.claude/skills/executing-plans/SKILL.md` - Review after each task completion - Fresh agent for unbiased review diff --git a/.claude/agents/debugger.md b/.claude/agents/debugger.md index 82e0f86..cc51762 100644 --- a/.claude/agents/debugger.md +++ b/.claude/agents/debugger.md @@ -236,7 +236,7 @@ This agent works with: For enhanced systematic debugging, use the superpowers methodology: -**Reference**: `.claude/skills/methodology/systematic-debugging/SKILL.md` +**Reference**: `.claude/skills/systematic-debugging/SKILL.md` ### Four-Phase Methodology @@ -255,8 +255,8 @@ If 3+ consecutive fixes fail, STOP - this is an architectural problem. ### Additional Skills -- **Root cause tracing**: `.claude/skills/methodology/root-cause-tracing/SKILL.md` -- **Defense in depth**: `.claude/skills/methodology/defense-in-depth/SKILL.md` +- **Root cause tracing**: `.claude/skills/root-cause-tracing/SKILL.md` +- **Defense in depth**: `.claude/skills/defense-in-depth/SKILL.md` ## Project-Specific Overrides diff --git a/.claude/agents/planner.md b/.claude/agents/planner.md index b6f684f..ef45d30 100644 --- a/.claude/agents/planner.md +++ b/.claude/agents/planner.md @@ -144,7 +144,7 @@ Implement JWT-based authentication with login, logout, and token refresh capabil For enhanced detailed planning, use the superpowers methodology: -**Reference**: `.claude/skills/methodology/writing-plans/SKILL.md` +**Reference**: `.claude/skills/writing-plans/SKILL.md` ### Detailed Mode (2-5 min tasks) @@ -161,7 +161,7 @@ After creating a detailed plan: - **Subagent-driven**: Use `executing-plans` skill for automated execution - **Manual**: Developer follows plan sequentially -**Reference**: `.claude/skills/methodology/executing-plans/SKILL.md` +**Reference**: `.claude/skills/executing-plans/SKILL.md` ## Project-Specific Overrides diff --git a/.claude/agents/tester.md b/.claude/agents/tester.md index a8649bf..5b14b43 100644 --- a/.claude/agents/tester.md +++ b/.claude/agents/tester.md @@ -266,7 +266,7 @@ For enhanced testing practices, use the superpowers methodology: ### Test-Driven Development -**Reference**: `.claude/skills/methodology/test-driven-development/SKILL.md` +**Reference**: `.claude/skills/test-driven-development/SKILL.md` Key principles: - **NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST** @@ -277,7 +277,7 @@ Key principles: ### Verification -**Reference**: `.claude/skills/methodology/verification-before-completion/SKILL.md` +**Reference**: `.claude/skills/verification-before-completion/SKILL.md` Before claiming tests pass: 1. Identify the command that proves assertion @@ -288,7 +288,7 @@ Before claiming tests pass: ### Testing Anti-Patterns -**Reference**: `.claude/skills/methodology/testing-anti-patterns/SKILL.md` +**Reference**: `.claude/skills/testing-anti-patterns/SKILL.md` Avoid these mistakes: 1. Testing mock behavior instead of real code diff --git a/.claude/commands/brainstorm.md b/.claude/commands/brainstorm.md index b873a8f..24fdd38 100644 --- a/.claude/commands/brainstorm.md +++ b/.claude/commands/brainstorm.md @@ -20,7 +20,7 @@ Start interactive brainstorming session for: **$ARGUMENTS** ## Methodology -**Reference**: `.claude/skills/methodology/brainstorming/SKILL.md` +**Reference**: `.claude/skills/brainstorming/SKILL.md` This command uses the superpowers brainstorming methodology for optimal results. diff --git a/.claude/commands/execute-plan.md b/.claude/commands/execute-plan.md index 7d4eb5f..0a42076 100644 --- a/.claude/commands/execute-plan.md +++ b/.claude/commands/execute-plan.md @@ -20,7 +20,7 @@ Execute plan from: **$ARGUMENTS** ## Methodology -**Reference**: `.claude/skills/methodology/executing-plans/SKILL.md` +**Reference**: `.claude/skills/executing-plans/SKILL.md` This command uses the superpowers execution methodology for quality-gated implementation. @@ -104,7 +104,7 @@ After all tasks complete: 2. Review entire implementation against plan 3. Verify all success criteria met 4. Run full test suite -5. Use `finishing-development-branch` skill +5. Use `finishing-a-development-branch` skill ## Critical Rules diff --git a/.claude/commands/mode.md b/.claude/commands/mode.md index e18f164..ae790c2 100644 --- a/.claude/commands/mode.md +++ b/.claude/commands/mode.md @@ -14,7 +14,7 @@ Switch to the specified behavioral mode. |------|-------------|----------| | `default` | Balanced standard behavior | General tasks | | `brainstorm` | Creative exploration, more questions | Design, ideation | -| `token-efficient` | Compressed, concise output | High-volume, cost savings | +| `writing-concisely` | Compressed, concise output | High-volume, cost savings | | `deep-research` | Thorough analysis, citations | Investigation, audits | | `implementation` | Code-focused, minimal prose | Executing plans | | `review` | Critical analysis, finding issues | Code review, QA | diff --git a/.claude/commands/plan.md b/.claude/commands/plan.md index 87de153..f4de36b 100644 --- a/.claude/commands/plan.md +++ b/.claude/commands/plan.md @@ -201,7 +201,7 @@ Use `--detailed` flag for superpowers-style plans with 2-5 minute tasks: ### Detailed Mode Features -**Reference**: `.claude/skills/methodology/writing-plans/SKILL.md` +**Reference**: `.claude/skills/writing-plans/SKILL.md` When `--detailed` is specified: - **Bite-sized tasks**: 2-5 minutes each (vs standard 15-60 min) @@ -254,7 +254,7 @@ When `--detailed` is specified: Use `/execute-plan [plan-file]` for subagent-driven execution with code review gates. -**Reference**: `.claude/skills/methodology/executing-plans/SKILL.md` +**Reference**: `.claude/skills/executing-plans/SKILL.md` ## Flags diff --git a/.claude/commands/tdd.md b/.claude/commands/tdd.md index 137dca1..a454375 100644 --- a/.claude/commands/tdd.md +++ b/.claude/commands/tdd.md @@ -71,7 +71,7 @@ Add more test cases and repeat the cycle. ## Superpowers TDD Methodology -**Reference**: `.claude/skills/methodology/test-driven-development/SKILL.md` +**Reference**: `.claude/skills/test-driven-development/SKILL.md` ### Non-Negotiable Rule @@ -90,7 +90,7 @@ RIGHT: Delete the code, write test, rewrite implementation ### Verification Before Completion -**Reference**: `.claude/skills/methodology/verification-before-completion/SKILL.md` +**Reference**: `.claude/skills/verification-before-completion/SKILL.md` Before claiming tests pass: 1. **Identify** the command that proves assertion @@ -108,7 +108,7 @@ Never use without verification: ### Testing Anti-Patterns to Avoid -**Reference**: `.claude/skills/methodology/testing-anti-patterns/SKILL.md` +**Reference**: `.claude/skills/testing-anti-patterns/SKILL.md` 1. Testing mock behavior instead of real code 2. Polluting production with test-only methods diff --git a/.claude/skills/api-client/SKILL.md b/.claude/skills/api-client/SKILL.md new file mode 100644 index 0000000..7383bde --- /dev/null +++ b/.claude/skills/api-client/SKILL.md @@ -0,0 +1,59 @@ +--- +name: api-client +description: > + Use when setting up axios, fetch, or httpx clients, implementing request interceptors, adding retry logic, handling authentication tokens, or generating type-safe API clients from OpenAPI specs. Also activate whenever code makes HTTP requests, integrates with external APIs, or needs robust error handling for network calls. +--- + +# API Client Patterns + +## When to Use + +- Setting up HTTP clients (httpx, axios, fetch) with base URLs and default headers +- Implementing request/response interceptors for auth tokens, logging, or error transformation +- Adding retry logic with exponential backoff for transient failures +- Generating type-safe API clients from OpenAPI specifications +- Handling authentication tokens (Bearer, API key) in outbound requests +- Building wrapper classes around third-party REST APIs + +## When NOT to Use + +- Building API servers (use `backend-frameworks`) +- Making simple one-off HTTP calls that don't need a configured client +- GraphQL clients (use Apollo or urql documentation directly) + +--- + +## Quick Reference + +| Topic | Reference | Key content | +|-------|-----------|-------------| +| All client patterns | `references/patterns.md` | httpx, axios, fetch wrappers, interceptors, retry, type-safe clients | +| HTTP client recipes | `references/http-client-patterns.md` | Advanced patterns, streaming, file uploads | + +--- + +## Best Practices + +1. **Create a single configured client instance.** Don't construct new clients per request. Configure base URL, timeouts, and default headers once. +2. **Set explicit timeouts.** Never use default (infinite) timeouts. Set connect and read timeouts separately. +3. **Use interceptors for cross-cutting concerns.** Auth token injection, request logging, and error transformation belong in interceptors, not in each call site. +4. **Implement retry with exponential backoff and jitter.** Only retry idempotent requests (GET, PUT, DELETE) and transient errors (5xx, network errors). +5. **Respect Retry-After headers.** When the server sends `Retry-After`, honor it instead of using your own backoff schedule. +6. **Generate clients from OpenAPI specs.** Use `openapi-typescript` + `openapi-fetch` (TypeScript) or `openapi-python-client` (Python) for type-safe API consumption. +7. **Handle errors at the boundary.** Transform HTTP errors into domain-specific errors at the client wrapper level. + +## Common Pitfalls + +1. **No timeout configured** — requests hang indefinitely on unresponsive servers. +2. **Retrying non-idempotent requests** — retrying POST can create duplicates. Use idempotency keys. +3. **Swallowing error details** — wrapping errors without preserving the original status code and message. +4. **Token refresh race conditions** — multiple concurrent requests all try to refresh the token simultaneously. Use a mutex/lock. +5. **Not closing client connections** — httpx AsyncClient and axios instances should be properly closed/disposed. + +--- + +## Related Skills + +- `error-handling` — Error transformation and retry patterns +- `authentication` — Token management for API calls +- `backend-frameworks` — Building the APIs these clients consume diff --git a/.claude/skills/patterns/api-client/references/http-client-patterns.md b/.claude/skills/api-client/references/http-client-patterns.md similarity index 100% rename from .claude/skills/patterns/api-client/references/http-client-patterns.md rename to .claude/skills/api-client/references/http-client-patterns.md diff --git a/.claude/skills/patterns/api-client/SKILL.md b/.claude/skills/api-client/references/patterns.md similarity index 96% rename from .claude/skills/patterns/api-client/SKILL.md rename to .claude/skills/api-client/references/patterns.md index 47c3e82..8b9f7b1 100644 --- a/.claude/skills/patterns/api-client/SKILL.md +++ b/.claude/skills/api-client/references/patterns.md @@ -1,8 +1,5 @@ ---- -name: api-client -description: > - HTTP client patterns for consuming REST APIs in Python and TypeScript. Use this skill when setting up axios, fetch, or httpx clients, implementing request interceptors, adding retry logic, handling authentication tokens, or generating type-safe API clients from OpenAPI specs. Trigger whenever code makes HTTP requests, integrates with external APIs, or needs robust error handling for network calls. ---- +# Api Client — Patterns + # API Client Patterns @@ -765,8 +762,8 @@ class ThrottledClient: ## Related Skills -- `api/openapi` - OpenAPI spec design and documentation -- `patterns/error-handling` - Structured error handling patterns across the stack -- `patterns/authentication` - Authentication token management and OAuth2 flows -- `patterns/caching` - HTTP caching, conditional requests, and cache invalidation -- `patterns/logging` - Logging HTTP requests and responses +- `openapi` - OpenAPI spec design and documentation +- `error-handling` - Structured error handling patterns across the stack +- `authentication` - Authentication token management and OAuth2 flows +- `caching` - HTTP caching, conditional requests, and cache invalidation +- `logging` - Logging HTTP requests and responses diff --git a/.claude/skills/api/openapi/SKILL.md b/.claude/skills/api/openapi/SKILL.md deleted file mode 100644 index 575b069..0000000 --- a/.claude/skills/api/openapi/SKILL.md +++ /dev/null @@ -1,837 +0,0 @@ ---- -name: openapi -description: > - Use this skill when designing, documenting, or generating REST API specifications using OpenAPI/Swagger. Trigger on keywords like OpenAPI, Swagger, API spec, REST documentation, API schema, request body, response schema, and API client generation. Also apply when adopting design-first API development, validating API contracts, or setting up auto-generated API documentation for FastAPI, Express, or NestJS endpoints. ---- - -# OpenAPI & REST API Design - -## When to Use - -- Documenting REST APIs -- Generating API clients -- API design-first development -- Defining webhook contracts -- Establishing pagination, versioning, or auth patterns for a new service - -## When NOT to Use - -- Internal-only scripts or automation that do not expose HTTP endpoints -- CLI tools and command-line utilities without a REST interface -- GraphQL APIs where a different specification format applies - ---- - -## Core Patterns - -### 1. OpenAPI 3.1 Specification Structure - -A complete spec skeleton showing every top-level section. Use `$ref` to split -large specs into per-resource files. - -```yaml -openapi: 3.1.0 -info: - title: Acme API - version: 2.0.0 - description: Public API for the Acme platform. - contact: - name: API Support - email: api@acme.dev - license: - name: MIT - url: https://opensource.org/licenses/MIT - -servers: - - url: https://api.acme.dev/v2 - description: Production - - url: https://staging-api.acme.dev/v2 - description: Staging - -tags: - - name: Users - description: User management operations - - name: Orders - description: Order lifecycle operations - -paths: - /users: - $ref: './paths/users.yaml' - /users/{userId}: - $ref: './paths/users-by-id.yaml' - /orders: - $ref: './paths/orders.yaml' - -components: - schemas: - $ref: './components/schemas/_index.yaml' - securitySchemes: - BearerAuth: - type: http - scheme: bearer - bearerFormat: JWT - ApiKeyAuth: - type: apiKey - in: header - name: X-API-Key - -security: - - BearerAuth: [] - -webhooks: - orderCompleted: - $ref: './webhooks/order-completed.yaml' -``` - -**Organizing with `$ref`** -- keep one file per resource under `paths/` and -shared schemas under `components/schemas/`. A bundler such as -`@redocly/cli bundle` resolves references into a single file for tooling. - -``` -spec/ -├── openapi.yaml # Root document -├── paths/ -│ ├── users.yaml -│ ├── users-by-id.yaml -│ └── orders.yaml -├── components/ -│ └── schemas/ -│ ├── _index.yaml -│ ├── User.yaml -│ ├── Order.yaml -│ └── ProblemDetail.yaml -└── webhooks/ - └── order-completed.yaml -``` - ---- - -### 2. Path & Operation Patterns - -#### RESTful URL Naming Conventions - -- Use **plural nouns** for collections: `/users`, `/orders`. -- Use **path parameters** for single-resource access: `/users/{userId}`. -- Nest only one level deep: `/users/{userId}/orders` (not deeper). -- Use **query parameters** for filtering, sorting, and pagination. -- Avoid verbs in paths -- let HTTP methods convey the action. - -#### CRUD Operations - -```yaml -paths: - /users: - get: - operationId: listUsers - tags: [Users] - summary: List users - parameters: - - $ref: '#/components/parameters/PageCursor' - - $ref: '#/components/parameters/PageSize' - - name: status - in: query - schema: - type: string - enum: [active, inactive] - responses: - '200': - description: Paginated list of users - content: - application/json: - schema: - $ref: '#/components/schemas/UserListResponse' - - post: - operationId: createUser - tags: [Users] - summary: Create a user - requestBody: - required: true - content: - application/json: - schema: - $ref: '#/components/schemas/CreateUserRequest' - responses: - '201': - description: User created - headers: - Location: - schema: - type: string - description: URL of the new resource - content: - application/json: - schema: - $ref: '#/components/schemas/User' - '409': - $ref: '#/components/responses/Conflict' - '422': - $ref: '#/components/responses/ValidationError' - - /users/{userId}: - parameters: - - name: userId - in: path - required: true - schema: - type: string - format: uuid - - get: - operationId: getUser - tags: [Users] - summary: Get a single user - responses: - '200': - description: User found - content: - application/json: - schema: - $ref: '#/components/schemas/User' - '404': - $ref: '#/components/responses/NotFound' - - patch: - operationId: updateUser - tags: [Users] - summary: Partially update a user - requestBody: - required: true - content: - application/json: - schema: - $ref: '#/components/schemas/UpdateUserRequest' - responses: - '200': - description: User updated - content: - application/json: - schema: - $ref: '#/components/schemas/User' - '404': - $ref: '#/components/responses/NotFound' - '422': - $ref: '#/components/responses/ValidationError' - - delete: - operationId: deleteUser - tags: [Users] - summary: Delete a user - responses: - '204': - description: User deleted - '404': - $ref: '#/components/responses/NotFound' -``` - -#### Path Parameters vs Query Parameters - -| Use case | Mechanism | Example | -|----------|-----------|---------| -| Identify a specific resource | Path parameter | `/orders/{orderId}` | -| Filter a collection | Query parameter | `/orders?status=shipped` | -| Sort a collection | Query parameter | `/orders?sort=-createdAt` | -| Paginate | Query parameter | `/orders?cursor=abc&limit=20` | -| Expand nested data | Query parameter | `/orders?expand=items,customer` | - ---- - -### 3. Request Body Patterns - -#### JSON Request Body with Validation - -```yaml -components: - schemas: - CreateUserRequest: - type: object - required: - - email - - name - properties: - email: - type: string - format: email - maxLength: 254 - name: - type: string - minLength: 1 - maxLength: 100 - role: - type: string - enum: [admin, member, viewer] - default: member - additionalProperties: false -``` - -Implementation in **FastAPI** (Python): - -```python -from pydantic import BaseModel, EmailStr, Field - -class CreateUserRequest(BaseModel): - email: EmailStr - name: str = Field(min_length=1, max_length=100) - role: str = Field(default="member", pattern="^(admin|member|viewer)$") - - model_config = {"extra": "forbid"} -``` - -Implementation in **Express** (TypeScript with Zod): - -```typescript -import { z } from "zod"; - -const CreateUserRequest = z.object({ - email: z.string().email().max(254), - name: z.string().min(1).max(100), - role: z.enum(["admin", "member", "viewer"]).default("member"), -}).strict(); - -type CreateUserRequest = z.infer; -``` - -#### Multipart Form Data (File Uploads) - -```yaml -/users/{userId}/avatar: - put: - operationId: uploadAvatar - tags: [Users] - summary: Upload user avatar - parameters: - - name: userId - in: path - required: true - schema: - type: string - format: uuid - requestBody: - required: true - content: - multipart/form-data: - schema: - type: object - required: - - file - properties: - file: - type: string - format: binary - description: Image file (JPEG or PNG, max 5 MB) - caption: - type: string - maxLength: 200 - encoding: - file: - contentType: image/jpeg, image/png - responses: - '200': - description: Avatar updated - content: - application/json: - schema: - type: object - properties: - url: - type: string - format: uri - '413': - $ref: '#/components/responses/PayloadTooLarge' -``` - -#### Content Negotiation - -Support multiple response formats by listing them under `content`: - -```yaml -responses: - '200': - description: Export data - content: - application/json: - schema: - $ref: '#/components/schemas/ExportData' - text/csv: - schema: - type: string - application/pdf: - schema: - type: string - format: binary -``` - -Clients select a format with the `Accept` header. Document which formats your -API actually supports so consumers do not have to guess. - ---- - -### 4. Response Patterns - -#### Success Responses - -| Code | Meaning | Typical use | -|------|---------|-------------| -| `200` | OK | GET, PATCH, general success | -| `201` | Created | POST that creates a resource | -| `202` | Accepted | Async operation started | -| `204` | No Content | DELETE, or PUT with no body returned | - -Always return a `Location` header with `201` pointing to the new resource. - -#### Error Responses -- RFC 7807 Problem Details - -Define a single reusable error schema based on RFC 7807: - -```yaml -components: - schemas: - ProblemDetail: - type: object - required: - - type - - title - - status - properties: - type: - type: string - format: uri - description: URI reference identifying the problem type. - example: https://api.acme.dev/problems/validation-error - title: - type: string - description: Short human-readable summary. - example: Validation Error - status: - type: integer - description: HTTP status code. - example: 422 - detail: - type: string - description: Human-readable explanation specific to this occurrence. - example: "Field 'email' must be a valid email address." - instance: - type: string - format: uri - description: URI identifying this specific occurrence. - errors: - type: array - description: Field-level validation errors (optional extension). - items: - type: object - properties: - field: - type: string - example: email - message: - type: string - example: Must be a valid email address. - code: - type: string - example: invalid_format - - responses: - NotFound: - description: Resource not found - content: - application/problem+json: - schema: - $ref: '#/components/schemas/ProblemDetail' - example: - type: https://api.acme.dev/problems/not-found - title: Not Found - status: 404 - detail: User with ID '550e8400' was not found. - - ValidationError: - description: Request validation failed - content: - application/problem+json: - schema: - $ref: '#/components/schemas/ProblemDetail' - example: - type: https://api.acme.dev/problems/validation-error - title: Validation Error - status: 422 - errors: - - field: email - message: Must be a valid email address. - code: invalid_format - - Conflict: - description: Resource conflict - content: - application/problem+json: - schema: - $ref: '#/components/schemas/ProblemDetail' - - PayloadTooLarge: - description: Request payload exceeds limit - content: - application/problem+json: - schema: - $ref: '#/components/schemas/ProblemDetail' -``` - -Use the `application/problem+json` media type for all error responses to signal -RFC 7807 compliance. - ---- - -### 5. Authentication Schemes - -#### Bearer Token (JWT) - -```yaml -components: - securitySchemes: - BearerAuth: - type: http - scheme: bearer - bearerFormat: JWT - -security: - - BearerAuth: [] -``` - -Override per-operation to allow unauthenticated access: - -```yaml -paths: - /health: - get: - security: [] # No auth required - responses: - '200': - description: Healthy -``` - -#### API Key - -```yaml -components: - securitySchemes: - ApiKeyHeader: - type: apiKey - in: header - name: X-API-Key - ApiKeyQuery: - type: apiKey - in: query - name: api_key -``` - -#### OAuth2 Flows - -```yaml -components: - securitySchemes: - OAuth2: - type: oauth2 - flows: - authorizationCode: - authorizationUrl: https://auth.acme.dev/authorize - tokenUrl: https://auth.acme.dev/token - refreshUrl: https://auth.acme.dev/token - scopes: - users:read: Read user profiles - users:write: Create and update users - orders:read: Read orders - -paths: - /users: - get: - security: - - OAuth2: [users:read] -``` - ---- - -### 6. Pagination Patterns - -#### Cursor-Based Pagination (Recommended) - -Best for large, real-time datasets where rows may be inserted or deleted -between pages. - -```yaml -components: - parameters: - PageCursor: - name: cursor - in: query - description: Opaque cursor returned by a previous response. - schema: - type: string - PageSize: - name: limit - in: query - description: Maximum items per page. - schema: - type: integer - minimum: 1 - maximum: 100 - default: 20 - - schemas: - UserListResponse: - type: object - required: - - data - - pagination - properties: - data: - type: array - items: - $ref: '#/components/schemas/User' - pagination: - type: object - required: - - hasMore - properties: - nextCursor: - type: string - nullable: true - hasMore: - type: boolean -``` - -#### Offset-Based Pagination - -Simpler but less efficient for large tables and susceptible to drift when data -changes between requests. - -```yaml -components: - parameters: - PageOffset: - name: offset - in: query - schema: - type: integer - minimum: 0 - default: 0 - PageLimit: - name: limit - in: query - schema: - type: integer - minimum: 1 - maximum: 100 - default: 20 - - schemas: - PaginatedResponse: - type: object - required: - - data - - total - - offset - - limit - properties: - data: - type: array - items: {} - total: - type: integer - description: Total number of matching records. - offset: - type: integer - limit: - type: integer -``` - -#### Response Envelope Pattern - -Wrap every collection in a consistent envelope so clients always know where to -find the data and metadata: - -```json -{ - "data": [ ... ], - "pagination": { "nextCursor": "abc123", "hasMore": true }, - "meta": { "requestId": "req_xyz", "timestamp": "2026-03-29T12:00:00Z" } -} -``` - ---- - -### 7. API Versioning - -#### URL Versioning - -```yaml -servers: - - url: https://api.acme.dev/v1 - description: Version 1 (deprecated) - - url: https://api.acme.dev/v2 - description: Version 2 (current) -``` - -Pros: explicit, easy to route, cache-friendly. -Cons: duplicates paths across versions, harder to share schemas. - -#### Header Versioning - -```yaml -parameters: - - name: X-API-Version - in: header - required: false - schema: - type: string - enum: ['2024-01-15', '2025-06-01'] - default: '2025-06-01' - description: Date-based API version. Defaults to latest stable. -``` - -Pros: clean URLs, fine-grained control. -Cons: less discoverable, harder to test in a browser. - -#### Trade-offs Summary - -| Approach | Discoverability | URL cleanliness | Caching | Migration effort | -|----------|----------------|-----------------|---------|-----------------| -| URL path | High | Lower | Easy | Higher (path changes) | -| Header | Lower | High | Needs Vary header | Lower | -| Query param | Medium | Medium | Easy | Lower | - -Pick one approach and use it consistently. URL versioning is the most common -choice for public APIs; header versioning suits internal services. - ---- - -### 8. Webhook Specifications - -OpenAPI 3.1 supports a top-level `webhooks` key for documenting outbound -event payloads your API will send to consumer-registered URLs. - -```yaml -webhooks: - orderCompleted: - post: - operationId: onOrderCompleted - summary: Fired when an order reaches "completed" status. - requestBody: - required: true - content: - application/json: - schema: - $ref: '#/components/schemas/OrderCompletedEvent' - responses: - '200': - description: Webhook received successfully. - -components: - schemas: - WebhookEventBase: - type: object - required: - - id - - type - - createdAt - properties: - id: - type: string - format: uuid - type: - type: string - createdAt: - type: string - format: date-time - - OrderCompletedEvent: - allOf: - - $ref: '#/components/schemas/WebhookEventBase' - - type: object - required: - - data - properties: - type: - type: string - const: order.completed - data: - type: object - properties: - orderId: - type: string - format: uuid - total: - type: number - format: double - currency: - type: string - example: USD -``` - -Document a shared `WebhookEventBase` so all event payloads have a consistent -envelope with `id`, `type`, and `createdAt`. - ---- - -## Best Practices - -1. **Use consistent, plural resource names.** `/users`, `/orders`, `/invoices` - -- never mix singular and plural within the same API. - -2. **Make mutating operations idempotent.** Accept an `Idempotency-Key` header - on POST endpoints so clients can safely retry without creating duplicates. - -3. **Return rate-limit headers on every response.** Include `X-RateLimit-Limit`, - `X-RateLimit-Remaining`, and `X-RateLimit-Reset` so clients can self-throttle. - -4. **Provide `operationId` for every operation.** Code generators use this as - the method name; without it, generated clients have meaningless names. - -5. **Include realistic examples in the spec.** Examples power documentation UIs, - mock servers, and contract tests. Add them at both the schema and operation - level. - -6. **Use `additionalProperties: false` on request schemas.** This catches typos - in client payloads early and prevents silently ignored fields. - -7. **Document hypermedia links (HATEOAS basics).** Even a minimal `_links` - object with `self` and `next` URIs helps clients navigate without hardcoding - paths. - -8. **Version your spec file alongside code.** Store the OpenAPI document in the - same repository as the implementation. Run a CI check (e.g., `redocly lint`) - to validate the spec on every pull request. - ---- - -## Common Pitfalls - -1. **Missing error documentation.** Every operation should list its possible - `4xx` and `5xx` responses. Consumers cannot handle errors they do not know - about. At minimum document `400`, `401`, `403`, `404`, and `500`. - -2. **Overusing `200 OK` for everything.** Return `201` for resource creation, - `204` for deletion, and `202` for asynchronous actions. Correct status codes - let generic HTTP clients behave properly (e.g., following `Location` headers). - -3. **Deeply nested resource URLs.** `/users/{uid}/orders/{oid}/items/{iid}/notes` - is fragile and hard to cache. Flatten to `/order-items/{iid}/notes` once the - relationship is established. - -4. **Inconsistent naming conventions.** Mixing `camelCase` and `snake_case` - within the same API confuses consumers. Pick one JSON field casing and enforce - it with a linter rule. - -5. **Ignoring `nullable` vs optional.** In OpenAPI 3.1, `nullable` is gone; - use `type: ["string", "null"]` instead. A field that is not in `required` - may be absent, but that is different from being explicitly `null`. Be precise - about which you intend. - -6. **No pagination on list endpoints.** Returning unbounded arrays will - eventually cause timeouts or OOM errors. Every collection endpoint should - accept `limit` and either `cursor` or `offset` from day one, even if the - dataset is currently small. - ---- - -## Related Skills - -- `patterns/api-client` - Patterns for consuming and generating API clients from specs -- `patterns/error-handling` - Consistent error response structures and handling -- `frameworks/fastapi` - FastAPI framework with built-in OpenAPI generation diff --git a/.claude/skills/api/openapi/templates/openapi-3.1-starter.yaml b/.claude/skills/api/openapi/templates/openapi-3.1-starter.yaml deleted file mode 100644 index f85df7e..0000000 --- a/.claude/skills/api/openapi/templates/openapi-3.1-starter.yaml +++ /dev/null @@ -1,240 +0,0 @@ -openapi: "3.1.0" -info: - title: My API - description: Starter API specification. Replace with your project details. - version: "1.0.0" - contact: - name: API Support - email: support@example.com - -servers: - - url: http://localhost:3000/api/v1 - description: Local development - - url: https://api.example.com/v1 - description: Production - -tags: - - name: Users - description: User management - - name: Health - description: Service health checks - -paths: - /health: - get: - tags: [Health] - summary: Health check - operationId: getHealth - responses: - "200": - description: Service is healthy - content: - application/json: - schema: - type: object - properties: - status: { type: string, example: ok } - timestamp: { type: string, format: date-time } - - /users: - get: - tags: [Users] - summary: List users - operationId: listUsers - security: [{ bearerAuth: [] }] - parameters: - - $ref: "#/components/parameters/PageParam" - - $ref: "#/components/parameters/PerPageParam" - - $ref: "#/components/parameters/SortParam" - - name: status - in: query - schema: { type: string, enum: [active, inactive] } - responses: - "200": - description: Paginated list of users - content: - application/json: - schema: - type: object - properties: - data: - type: array - items: { $ref: "#/components/schemas/User" } - pagination: { $ref: "#/components/schemas/Pagination" } - "401": { $ref: "#/components/responses/Unauthorized" } - - post: - tags: [Users] - summary: Create a user - operationId: createUser - security: [{ bearerAuth: [] }] - requestBody: - required: true - content: - application/json: - schema: { $ref: "#/components/schemas/CreateUserRequest" } - responses: - "201": - description: User created - headers: - Location: { schema: { type: string }, description: URL of created user } - content: - application/json: - schema: { $ref: "#/components/schemas/User" } - "400": { $ref: "#/components/responses/BadRequest" } - "401": { $ref: "#/components/responses/Unauthorized" } - "422": { $ref: "#/components/responses/ValidationError" } - - /users/{userId}: - parameters: - - name: userId - in: path - required: true - schema: { type: string } - - get: - tags: [Users] - summary: Get a user by ID - operationId: getUser - security: [{ bearerAuth: [] }] - responses: - "200": - description: User details - content: - application/json: - schema: { $ref: "#/components/schemas/User" } - "401": { $ref: "#/components/responses/Unauthorized" } - "404": { $ref: "#/components/responses/NotFound" } - - patch: - tags: [Users] - summary: Update a user - operationId: updateUser - security: [{ bearerAuth: [] }] - requestBody: - required: true - content: - application/json: - schema: { $ref: "#/components/schemas/UpdateUserRequest" } - responses: - "200": - description: User updated - content: - application/json: - schema: { $ref: "#/components/schemas/User" } - "401": { $ref: "#/components/responses/Unauthorized" } - "404": { $ref: "#/components/responses/NotFound" } - "422": { $ref: "#/components/responses/ValidationError" } - - delete: - tags: [Users] - summary: Delete a user - operationId: deleteUser - security: [{ bearerAuth: [] }] - responses: - "204": { description: User deleted } - "401": { $ref: "#/components/responses/Unauthorized" } - "404": { $ref: "#/components/responses/NotFound" } - -components: - securitySchemes: - bearerAuth: { type: http, scheme: bearer, bearerFormat: JWT } - apiKeyAuth: { type: apiKey, in: header, name: X-API-Key } - - parameters: - PageParam: - name: page - in: query - schema: { type: integer, minimum: 1, default: 1 } - PerPageParam: - name: per_page - in: query - schema: { type: integer, minimum: 1, maximum: 100, default: 25 } - SortParam: - name: sort - in: query - description: "Field to sort by. Prefix with - for descending." - schema: { type: string, example: "-created_at" } - - schemas: - User: - type: object - required: [id, email, name, status, created_at] - properties: - id: { type: string, example: usr_abc123 } - email: { type: string, format: email, example: jane@example.com } - name: { type: string, example: Jane Doe } - status: { type: string, enum: [active, inactive] } - created_at: { type: string, format: date-time } - updated_at: { type: string, format: date-time } - - CreateUserRequest: - type: object - required: [email, name] - properties: - email: { type: string, format: email } - name: { type: string, minLength: 1, maxLength: 100 } - role: { type: string, enum: [user, admin], default: user } - - UpdateUserRequest: - type: object - properties: - name: { type: string, minLength: 1, maxLength: 100 } - status: { type: string, enum: [active, inactive] } - - Pagination: - type: object - properties: - page: { type: integer } - per_page: { type: integer } - total: { type: integer } - total_pages: { type: integer } - - Error: - type: object - required: [error] - properties: - error: - type: object - required: [code, message] - properties: - code: { type: string } - message: { type: string } - details: - type: array - items: - type: object - properties: - field: { type: string } - message: { type: string } - request_id: { type: string } - - responses: - BadRequest: - description: Bad request - content: - application/json: - schema: { $ref: "#/components/schemas/Error" } - example: { error: { code: BAD_REQUEST, message: Malformed request body } } - Unauthorized: - description: Authentication required - content: - application/json: - schema: { $ref: "#/components/schemas/Error" } - example: { error: { code: UNAUTHORIZED, message: Missing or invalid token } } - NotFound: - description: Resource not found - content: - application/json: - schema: { $ref: "#/components/schemas/Error" } - example: { error: { code: NOT_FOUND, message: Resource not found } } - ValidationError: - description: Validation failed - content: - application/json: - schema: { $ref: "#/components/schemas/Error" } - example: - error: - code: VALIDATION_ERROR - message: Request validation failed - details: [{ field: email, message: Email already registered }] diff --git a/.claude/skills/authentication/SKILL.md b/.claude/skills/authentication/SKILL.md new file mode 100644 index 0000000..4a42e14 --- /dev/null +++ b/.claude/skills/authentication/SKILL.md @@ -0,0 +1,62 @@ +--- +name: authentication +description: > + Use when implementing JWT tokens, OAuth2 flows, session management, role-based access control (RBAC), password hashing, or multi-factor authentication. Also activate whenever code handles login, signup, token refresh, protected routes, permission checks, or user identity verification. Applies to middleware auth guards and API key authentication. +--- + +# Authentication & Authorization + +## When to Use + +- Implementing JWT creation, verification, and refresh token flows +- Building OAuth2 authorization code or PKCE flows +- Password hashing with argon2 or bcrypt +- Role-based access control (RBAC) or permission checks +- Session management with Redis or database-backed sessions +- API key authentication for service-to-service communication +- Multi-factor authentication (TOTP, SMS, email) + +## When NOT to Use + +- Token-free static sites or public APIs with no auth requirements +- Third-party auth services where implementation is fully managed (Auth0, Clerk) — unless customizing +- Simple scripts or CLI tools that do not need user identity + +--- + +## Quick Reference + +| Topic | Reference | Key content | +|-------|-----------|-------------| +| All auth patterns | `references/patterns.md` | JWT, OAuth2, password hashing, RBAC, sessions, API keys | +| Auth flow diagrams | `references/auth-flows.md` | Visual flow diagrams for OAuth2, JWT refresh, session lifecycle | + +--- + +## Best Practices + +1. **Never store passwords in plain text.** Use argon2id (preferred) or bcrypt with a work factor of 12+. +2. **Keep JWT tokens short-lived.** Access tokens should expire in 15-30 minutes. Use refresh tokens for longer sessions. +3. **Validate tokens on every request.** Never trust a token without verifying signature, expiration, and issuer. +4. **Use HttpOnly, Secure, SameSite cookies** for web session tokens. Never store tokens in localStorage. +5. **Implement token refresh rotation.** Invalidate old refresh tokens when a new one is issued to detect token theft. +6. **Separate authentication from authorization.** Auth verifies identity; authz checks permissions. Keep them in separate middleware/guards. +7. **Rate limit auth endpoints.** Login, registration, and password reset endpoints are prime brute-force targets. +8. **Log auth events.** Record login attempts (success and failure), token refreshes, and permission denials for security auditing. + +## Common Pitfalls + +1. **Storing JWTs in localStorage** — vulnerable to XSS. Use HttpOnly cookies instead. +2. **Not rotating refresh tokens** — a stolen refresh token gives permanent access. +3. **Hardcoding secrets** — JWT signing keys and API keys must come from environment variables. +4. **Missing token expiration checks** — always verify `exp` claim server-side. +5. **Overly broad RBAC roles** — prefer granular permissions over a few broad roles. +6. **Not hashing API keys** — store hashed API keys in the database, not plain text. + +--- + +## Related Skills + +- `owasp` — Security vulnerabilities in auth flows +- `backend-frameworks` — Framework-specific auth middleware +- `databases` — Storing user credentials and sessions diff --git a/.claude/skills/patterns/authentication/references/auth-flows.md b/.claude/skills/authentication/references/auth-flows.md similarity index 100% rename from .claude/skills/patterns/authentication/references/auth-flows.md rename to .claude/skills/authentication/references/auth-flows.md diff --git a/.claude/skills/patterns/authentication/SKILL.md b/.claude/skills/authentication/references/patterns.md similarity index 97% rename from .claude/skills/patterns/authentication/SKILL.md rename to .claude/skills/authentication/references/patterns.md index 698aeab..97292b2 100644 --- a/.claude/skills/patterns/authentication/SKILL.md +++ b/.claude/skills/authentication/references/patterns.md @@ -1,8 +1,5 @@ ---- -name: authentication -description: > - Authentication and authorization patterns for web applications. Use this skill when implementing JWT tokens, OAuth2 flows, session management, role-based access control (RBAC), password hashing, or multi-factor authentication. Trigger whenever code handles login, signup, token refresh, protected routes, permission checks, or user identity verification. Also applies to middleware auth guards and API key authentication. ---- +# Authentication — Patterns + # Authentication & Authorization Patterns @@ -853,7 +850,7 @@ async def verify_mfa(payload: MfaVerifyRequest, response: Response): ## Related Skills -- `security/owasp` - OWASP Top 10 security patterns and secure coding practices -- `patterns/api-client` - HTTP client patterns including auth token injection and refresh flows -- `frameworks/fastapi` - FastAPI-specific dependency injection and middleware patterns -- `frameworks/nextjs` - Next.js middleware and route protection patterns +- `owasp` - OWASP Top 10 security patterns and secure coding practices +- `api-client` - HTTP client patterns including auth token injection and refresh flows +- `fastapi` - FastAPI-specific dependency injection and middleware patterns +- `nextjs` - Next.js middleware and route protection patterns diff --git a/.claude/skills/backend-frameworks/SKILL.md b/.claude/skills/backend-frameworks/SKILL.md new file mode 100644 index 0000000..a493a16 --- /dev/null +++ b/.claude/skills/backend-frameworks/SKILL.md @@ -0,0 +1,65 @@ +--- +name: backend-frameworks +description: > + Use when building REST APIs or web servers with FastAPI, Django, NestJS, or Express — including routing, middleware, dependency injection, Pydantic models, serializers, controllers, services, guards, pipes, app.get, app.post, APIRouter, class-based views, or framework-specific patterns. +--- + +# Backend Frameworks + +## When to Use + +- Building REST APIs with FastAPI, Django REST Framework, NestJS, or Express +- Configuring middleware, routing, authentication, or request validation +- Setting up dependency injection, services, or module structure +- Integrating with databases via ORMs (SQLAlchemy, Django ORM, TypeORM, Prisma) +- WebSocket servers, microservices, or GraphQL resolvers + +## When NOT to Use + +- Frontend development — use `frontend` +- Database-specific queries without framework context — use `databases` +- API design/documentation — use `openapi` + +--- + +## Quick Reference + +| Framework | Reference | Language | Key features | +|-----------|-----------|----------|-------------| +| FastAPI | `references/fastapi.md` | Python | Pydantic, async, APIRouter, Depends(), OpenAPI auto-docs | +| Django | `references/django.md` | Python | ORM, admin, DRF serializers, class-based views, migrations | +| NestJS | `references/nestjs.md` | TypeScript | Modules, DI, guards, pipes, interceptors, Prisma/TypeORM | +| Express | `references/express.md` | TypeScript | Middleware, Router, error handling, helmet, rate limiting | + +--- + +## Best Practices + +1. **Use validation models for all request/response data.** Pydantic (FastAPI), class-validator DTOs (NestJS), Zod (Express), serializers (Django). +2. **Separate business logic from routes/controllers.** Route handlers handle HTTP; services handle domain logic. +3. **Organize routes by resource and version.** APIRouter (FastAPI), module structure (NestJS), Router (Express), URL conf (Django). +4. **Return proper HTTP status codes.** 201 for creation, 204 for deletion, 202 for accepted-but-not-done, 409 for conflicts. +5. **Use async all the way down.** Never mix sync blocking calls in async routes (especially FastAPI). +6. **Configure settings from environment variables.** pydantic-settings (FastAPI), django-environ (Django), dotenv (Express/NestJS). +7. **Use `select_related`/`prefetch_related` for every query touching relations** (Django). +8. **Use `transaction.atomic()` for multi-step writes** (Django). + +## Common Pitfalls + +1. **Blocking I/O in async routes.** `requests.get()`, `time.sleep()` in `async def` routes starves the event loop (FastAPI). +2. **Missing response_model / leaking internal fields** (FastAPI). +3. **N+1 queries from missing eager loading** (Django `select_related`, NestJS relations). +4. **Circular imports/dependencies.** Use `forwardRef()` (NestJS), restructure modules (Django/FastAPI). +5. **Forgetting `asyncHandler`** — unhandled promise rejections crash the process (Express). +6. **Error handler not registered last** (Express). +7. **Putting business logic in controllers** (NestJS). +8. **Not using `whitelist: true` on ValidationPipe** (NestJS). + +--- + +## Related Skills + +- `databases` — Database queries, schema design, migrations +- `openapi` — API specification and documentation +- `error-handling` — Exception handling and API error responses +- `authentication` — Auth flows for web applications diff --git a/.claude/skills/frameworks/django/SKILL.md b/.claude/skills/backend-frameworks/references/django.md similarity index 96% rename from .claude/skills/frameworks/django/SKILL.md rename to .claude/skills/backend-frameworks/references/django.md index d7ae068..c98f41f 100644 --- a/.claude/skills/frameworks/django/SKILL.md +++ b/.claude/skills/backend-frameworks/references/django.md @@ -1,8 +1,5 @@ ---- -name: django -description: > - Use this skill when working with Django web applications, Django ORM models, Django REST Framework APIs, or Django admin interfaces. Trigger for any mention of Django views, serializers, migrations, URL routing, class-based views, or Django middleware. Also applies when building Python web apps with templates, managing database schemas through Django migrations, or setting up admin panels. ---- +# Backend Frameworks — Django Patterns + # Django @@ -15,7 +12,7 @@ description: > ## When NOT to Use -- FastAPI projects — use the `frameworks/fastapi` skill instead for async APIs and microservices +- FastAPI projects — use the `fastapi` skill instead for async APIs and microservices - JavaScript/Node.js backends (Express, NestJS) — this skill is Python-only - Microservices architectures — consider FastAPI instead for lightweight, async services @@ -710,6 +707,6 @@ def transfer_project(project, new_owner): ## Related Skills -- `languages/python` — Python language patterns and best practices -- `databases/postgresql` — Database integration and query optimization -- `testing/pytest` — Testing Django applications with pytest-django +- `python` — Python language patterns and best practices +- `postgresql` — Database integration and query optimization +- `pytest` — Testing Django applications with pytest-django diff --git a/.claude/skills/backend-frameworks/references/express.md b/.claude/skills/backend-frameworks/references/express.md new file mode 100644 index 0000000..acbfee1 --- /dev/null +++ b/.claude/skills/backend-frameworks/references/express.md @@ -0,0 +1,451 @@ +# Backend Frameworks — Express Patterns + + +# Express + +## Overview + +Production patterns for building Node.js HTTP servers and REST APIs with Express. Covers routing, middleware, validation, error handling, authentication, database integration, and testing. + +## When to Use + +- Building REST APIs with Express (without NestJS) +- Adding middleware (auth, logging, rate limiting, CORS) +- Handling file uploads, streaming, or WebSockets on Express +- Migrating Express apps or adding features to existing ones + +## When NOT to Use + +- **NestJS projects** — use the `nestjs` skill (NestJS wraps Express but has its own patterns) +- **FastAPI / Django** — use the `fastapi` or `django` skill +- **Frontend** — use `react` or `nextjs` +- **Cloudflare Workers / edge** — use `cloudflare-workers` + +--- + +## Quick Reference + +| I need... | Go to | +|-----------|-------| +| Project structure | SS Architecture below | +| Route patterns | SS Routing below | +| Middleware | SS Middleware below | +| Input validation | SS Validation below | +| Error handling | SS Error Handling below | +| Auth patterns | SS Authentication below | +| Database integration | SS Database below | +| Testing | SS Testing below | + +--- + +## Architecture + +### Project structure + +``` +src/ +├── app.ts # Express app setup (middleware, routes) +├── server.ts # HTTP server bootstrap +├── routes/ +│ ├── index.ts # Route aggregator +│ ├── users.routes.ts # /api/users +│ └── orders.routes.ts # /api/orders +├── middleware/ +│ ├── auth.ts # JWT verification +│ ├── validate.ts # Zod validation middleware +│ ├── error-handler.ts # Global error handler +│ └── rate-limit.ts # Rate limiting +├── services/ +│ ├── users.service.ts # Business logic +│ └── orders.service.ts +├── models/ # Prisma or TypeORM entities +├── utils/ +│ └── async-handler.ts # Async error wrapper +└── tests/ + ├── users.test.ts + └── orders.test.ts +``` + +### App setup + +```typescript +// src/app.ts +import express from 'express'; +import helmet from 'helmet'; +import cors from 'cors'; +import { json } from 'express'; +import { router } from './routes'; +import { errorHandler } from './middleware/error-handler'; + +const app = express(); + +// Security middleware +app.use(helmet()); +app.use(cors({ origin: process.env.ALLOWED_ORIGINS?.split(',') })); + +// Body parsing +app.use(json({ limit: '10kb' })); + +// Routes +app.use('/api', router); + +// Health check +app.get('/health', (_req, res) => res.json({ status: 'ok' })); + +// Global error handler (must be last) +app.use(errorHandler); + +export { app }; +``` + +```typescript +// src/server.ts +import { app } from './app'; + +const PORT = process.env.PORT ?? 3000; +app.listen(PORT, () => console.log(`Listening on :${PORT}`)); +``` + +--- + +## Routing + +### Router pattern + +```typescript +// src/routes/users.routes.ts +import { Router } from 'express'; +import { UsersService } from '../services/users.service'; +import { validate } from '../middleware/validate'; +import { createUserSchema, updateUserSchema } from '../schemas/user.schema'; +import { asyncHandler } from '../utils/async-handler'; + +const router = Router(); +const service = new UsersService(); + +router.post('/', validate(createUserSchema), asyncHandler(async (req, res) => { + const user = await service.create(req.body); + res.status(201).json(user); +})); + +router.get('/:id', asyncHandler(async (req, res) => { + const user = await service.findOne(req.params.id); + if (!user) { + res.status(404).json({ type: 'not-found', title: 'Not Found', status: 404, detail: `User ${req.params.id} not found` }); + return; + } + res.json(user); +})); + +router.patch('/:id', validate(updateUserSchema), asyncHandler(async (req, res) => { + const user = await service.update(req.params.id, req.body); + res.json(user); +})); + +router.delete('/:id', asyncHandler(async (req, res) => { + await service.remove(req.params.id); + res.status(204).end(); +})); + +export { router as usersRouter }; +``` + +### Async error wrapper + +```typescript +// src/utils/async-handler.ts +import { Request, Response, NextFunction, RequestHandler } from 'express'; + +export function asyncHandler( + fn: (req: Request, res: Response, next: NextFunction) => Promise +): RequestHandler { + return (req, res, next) => fn(req, res, next).catch(next); +} +``` + +### Route aggregator + +```typescript +// src/routes/index.ts +import { Router } from 'express'; +import { usersRouter } from './users.routes'; +import { ordersRouter } from './orders.routes'; + +const router = Router(); +router.use('/users', usersRouter); +router.use('/orders', ordersRouter); + +export { router }; +``` + +--- + +## Middleware + +### Middleware order matters + +```typescript +// Correct order in app.ts: +app.use(helmet()); // 1. Security headers +app.use(cors()); // 2. CORS +app.use(json()); // 3. Body parsing +app.use(requestLogger); // 4. Logging +app.use(rateLimiter); // 5. Rate limiting +app.use('/api', router); // 6. Routes +app.use(errorHandler); // 7. Error handler (MUST be last) +``` + +### Request logging + +```typescript +import { Request, Response, NextFunction } from 'express'; + +export function requestLogger(req: Request, res: Response, next: NextFunction) { + const start = Date.now(); + res.on('finish', () => { + console.log(`${req.method} ${req.originalUrl} ${res.statusCode} ${Date.now() - start}ms`); + }); + next(); +} +``` + +### Rate limiting + +```typescript +import rateLimit from 'express-rate-limit'; + +export const apiLimiter = rateLimit({ + windowMs: 15 * 60 * 1000, // 15 minutes + max: 100, // 100 requests per window + standardHeaders: true, + legacyHeaders: false, + message: { type: 'rate-limit', title: 'Too Many Requests', status: 429 }, +}); +``` + +--- + +## Validation + +### Zod validation middleware + +```typescript +// src/middleware/validate.ts +import { Request, Response, NextFunction } from 'express'; +import { ZodSchema, ZodError } from 'zod'; + +export function validate(schema: ZodSchema) { + return (req: Request, res: Response, next: NextFunction) => { + try { + req.body = schema.parse(req.body); + next(); + } catch (err) { + if (err instanceof ZodError) { + res.status(400).json({ + type: 'validation-error', + title: 'Bad Request', + status: 400, + detail: err.errors.map(e => `${e.path.join('.')}: ${e.message}`).join('; '), + }); + return; + } + next(err); + } + }; +} +``` + +```typescript +// src/schemas/user.schema.ts +import { z } from 'zod'; + +export const createUserSchema = z.object({ + email: z.string().email().max(254), + name: z.string().min(1).max(100), + role: z.enum(['admin', 'member', 'viewer']).default('member'), +}); + +export const updateUserSchema = createUserSchema.partial(); + +export type CreateUserInput = z.infer; +``` + +--- + +## Error Handling + +### Global error handler (RFC 9457 Problem Details) + +```typescript +// src/middleware/error-handler.ts +import { Request, Response, NextFunction } from 'express'; + +export class AppError extends Error { + constructor(public statusCode: number, message: string) { + super(message); + } +} + +export function errorHandler(err: Error, _req: Request, res: Response, _next: NextFunction) { + const status = err instanceof AppError ? err.statusCode : 500; + const title = status >= 500 ? 'Internal Server Error' : err.message; + + if (status >= 500) console.error(err); + + res.status(status).json({ + type: `https://api.example.com/problems/${status}`, + title, + status, + detail: status >= 500 ? 'An unexpected error occurred' : err.message, + }); +} +``` + +--- + +## Authentication + +### JWT middleware + +```typescript +// src/middleware/auth.ts +import { Request, Response, NextFunction } from 'express'; +import jwt from 'jsonwebtoken'; + +export interface AuthRequest extends Request { + user?: { sub: string; role: string }; +} + +export function authenticate(req: AuthRequest, res: Response, next: NextFunction) { + const token = req.headers.authorization?.replace('Bearer ', ''); + if (!token) { + res.status(401).json({ type: 'unauthorized', title: 'Unauthorized', status: 401, detail: 'Missing bearer token' }); + return; + } + + try { + const payload = jwt.verify(token, process.env.JWT_SECRET!) as AuthRequest['user']; + req.user = payload; + next(); + } catch { + res.status(401).json({ type: 'unauthorized', title: 'Unauthorized', status: 401, detail: 'Invalid or expired token' }); + } +} + +export function authorize(...roles: string[]) { + return (req: AuthRequest, res: Response, next: NextFunction) => { + if (!req.user || !roles.includes(req.user.role)) { + res.status(403).json({ type: 'forbidden', title: 'Forbidden', status: 403, detail: 'Insufficient permissions' }); + return; + } + next(); + }; +} +``` + +--- + +## Database + +### Prisma integration + +```typescript +// src/db.ts +import { PrismaClient } from '@prisma/client'; + +export const prisma = new PrismaClient(); + +// Graceful shutdown +process.on('SIGTERM', async () => { + await prisma.$disconnect(); + process.exit(0); +}); +``` + +```typescript +// src/services/users.service.ts +import { prisma } from '../db'; +import { CreateUserInput } from '../schemas/user.schema'; + +export class UsersService { + async findOne(id: string) { + return prisma.user.findUnique({ where: { id } }); + } + + async create(data: CreateUserInput) { + return prisma.user.create({ data }); + } + + async update(id: string, data: Partial) { + return prisma.user.update({ where: { id }, data }); + } + + async remove(id: string) { + await prisma.user.delete({ where: { id } }); + } +} +``` + +--- + +## Testing + +### Integration tests with supertest + +```typescript +// src/tests/users.test.ts +import request from 'supertest'; +import { app } from '../app'; +import { prisma } from '../db'; + +describe('Users API', () => { + afterAll(() => prisma.$disconnect()); + + it('POST /api/users — creates user', async () => { + const res = await request(app) + .post('/api/users') + .send({ email: 'test@example.com', name: 'Test' }) + .expect(201); + + expect(res.body).toHaveProperty('id'); + expect(res.body.email).toBe('test@example.com'); + }); + + it('POST /api/users — rejects invalid email', async () => { + await request(app) + .post('/api/users') + .send({ email: 'not-an-email', name: 'Test' }) + .expect(400); + }); + + it('GET /api/users/:id — returns 404 for missing user', async () => { + await request(app) + .get('/api/users/nonexistent') + .expect(404); + }); +}); +``` + +--- + +## Common Pitfalls + +1. **Forgetting `asyncHandler`.** Unhandled promise rejections crash the process. Wrap every async route handler. +2. **Error handler not last.** Express error handlers must have 4 parameters `(err, req, res, next)` and must be registered after all routes. +3. **Not calling `next()`.** Middleware that doesn't call `next()` or send a response will hang the request. +4. **Mutating `req.body` without validation.** Always validate before trusting input. Use Zod or Joi middleware. +5. **Hardcoding CORS origin.** Use environment variables for allowed origins. Never use `cors({ origin: '*' })` in production. +6. **Missing `helmet()`.** Always use helmet for security headers. It's one line and prevents common attacks. +7. **Not limiting body size.** Use `json({ limit: '10kb' })` to prevent denial-of-service via large payloads. +8. **Using `express.static` for uploads.** Serve user uploads from a CDN or S3, not from the Express process. + +--- + +## Related Skills + +- `nestjs` — If you need DI, decorators, and modules, use NestJS instead of raw Express +- `openapi` — OpenAPI spec design for Express APIs (use `swagger-jsdoc` + `swagger-ui-express`) +- `typescript` — TypeScript patterns (Express is typed via `@types/express`) +- `docker` — Containerizing Express apps +- `authentication` — JWT / OAuth2 patterns (framework-agnostic) +- `jest` — Testing Express with Jest + supertest diff --git a/.claude/skills/frameworks/fastapi/SKILL.md b/.claude/skills/backend-frameworks/references/fastapi.md similarity index 95% rename from .claude/skills/frameworks/fastapi/SKILL.md rename to .claude/skills/backend-frameworks/references/fastapi.md index b099a60..4a988eb 100644 --- a/.claude/skills/frameworks/fastapi/SKILL.md +++ b/.claude/skills/backend-frameworks/references/fastapi.md @@ -1,8 +1,5 @@ ---- -name: fastapi -description: > - Use this skill when building REST APIs with Python and FastAPI, creating async web applications, or generating OpenAPI/Swagger documentation. Trigger for any mention of FastAPI, Pydantic models, async Python endpoints, dependency injection in Python APIs, or APIRouter patterns. Also applies when setting up Python microservices, adding request validation with Pydantic, or configuring ASGI applications. ---- +# Backend Frameworks — FastAPI Patterns + # FastAPI @@ -16,7 +13,7 @@ description: > ## When NOT to Use -- Django projects — use the `frameworks/django` skill instead +- Django projects — use the `django` skill instead - JavaScript/Node.js backends (Express, NestJS) — this skill is Python-only - Non-API applications such as CLI tools, desktop apps, or batch processing scripts @@ -672,9 +669,9 @@ settings = Settings() ## Related Skills -- `languages/python` — Python language patterns and best practices -- `api/openapi` — OpenAPI specification and documentation standards -- `databases/postgresql` — Database integration with async SQLAlchemy -- `testing/pytest` — Testing FastAPI applications with pytest and httpx -- `patterns/authentication` — JWT, OAuth2, and session patterns for FastAPI endpoints -- `patterns/logging` — Structured logging for FastAPI applications +- `python` — Python language patterns and best practices +- `openapi` — OpenAPI specification and documentation standards +- `postgresql` — Database integration with async SQLAlchemy +- `pytest` — Testing FastAPI applications with pytest and httpx +- `authentication` — JWT, OAuth2, and session patterns for FastAPI endpoints +- `logging` — Structured logging for FastAPI applications diff --git a/.claude/skills/backend-frameworks/references/nestjs.md b/.claude/skills/backend-frameworks/references/nestjs.md new file mode 100644 index 0000000..a78cbb1 --- /dev/null +++ b/.claude/skills/backend-frameworks/references/nestjs.md @@ -0,0 +1,660 @@ +# Backend Frameworks — NestJS Patterns + + +# NestJS + +## Overview + +Production patterns for building TypeScript backend APIs with NestJS. Covers module architecture, dependency injection, request validation, authentication guards, database integration, testing, and deployment. + +## When to Use +- Building REST APIs or GraphQL servers with NestJS +- Configuring modules, providers, and dependency injection +- Creating guards, interceptors, pipes, or middleware +- Integrating Prisma, TypeORM, or MikroORM with NestJS +- Building microservices or WebSocket gateways +- Testing NestJS controllers and services + +## When NOT to Use +- **Express without NestJS** — use `express` patterns directly +- **FastAPI / Django** — use the `fastapi` or `django` skill +- **Frontend** — use `react` or `nextjs` +- **Simple scripts** — NestJS is overkill for one-file utilities + +--- + +## Quick Reference + +| I need... | Go to | +|-----------|-------| +| Module/DI patterns | § Architecture below | +| Request validation | § Pipes & Validation below | +| Auth guards (JWT/API key) | § Authentication below | +| Database integration | § Database below | +| Testing patterns | § Testing below | +| Error handling | § Exception Filters below | +| OpenAPI generation | § OpenAPI below | + +--- + +## Architecture + +### Module structure + +Every NestJS app is a tree of modules. Keep modules focused on a single domain. + +``` +src/ +├── app.module.ts # Root module — imports feature modules +├── main.ts # Bootstrap +├── common/ # Shared utilities +│ ├── decorators/ +│ ├── filters/ +│ ├── guards/ +│ ├── interceptors/ +│ └── pipes/ +├── config/ # Configuration module +│ ├── config.module.ts +│ └── config.service.ts +├── users/ # Feature module +│ ├── users.module.ts +│ ├── users.controller.ts +│ ├── users.service.ts +│ ├── dto/ +│ │ ├── create-user.dto.ts +│ │ └── update-user.dto.ts +│ ├── entities/ +│ │ └── user.entity.ts +│ └── users.controller.spec.ts +└── orders/ # Another feature module + ├── orders.module.ts + ├── orders.controller.ts + └── orders.service.ts +``` + +### Module pattern + +```typescript +// users/users.module.ts +import { Module } from '@nestjs/common'; +import { UsersController } from './users.controller'; +import { UsersService } from './users.service'; + +@Module({ + controllers: [UsersController], + providers: [UsersService], + exports: [UsersService], // Expose to other modules +}) +export class UsersModule {} +``` + +### Controller + Service pattern + +```typescript +// users/users.controller.ts +import { Controller, Get, Post, Body, Param, Patch, Delete, HttpCode, HttpStatus } from '@nestjs/common'; +import { UsersService } from './users.service'; +import { CreateUserDto } from './dto/create-user.dto'; +import { UpdateUserDto } from './dto/update-user.dto'; + +@Controller('users') +export class UsersController { + constructor(private readonly usersService: UsersService) {} + + @Post() + @HttpCode(HttpStatus.CREATED) + create(@Body() dto: CreateUserDto) { + return this.usersService.create(dto); + } + + @Get(':id') + findOne(@Param('id') id: string) { + return this.usersService.findOne(id); + } + + @Patch(':id') + update(@Param('id') id: string, @Body() dto: UpdateUserDto) { + return this.usersService.update(id, dto); + } + + @Delete(':id') + @HttpCode(HttpStatus.NO_CONTENT) + remove(@Param('id') id: string) { + return this.usersService.remove(id); + } +} +``` + +```typescript +// users/users.service.ts +import { Injectable, NotFoundException } from '@nestjs/common'; + +@Injectable() +export class UsersService { + async findOne(id: string) { + const user = await this.prisma.user.findUnique({ where: { id } }); + if (!user) throw new NotFoundException(`User ${id} not found`); + return user; + } + + async create(dto: CreateUserDto) { + return this.prisma.user.create({ data: dto }); + } + + async update(id: string, dto: UpdateUserDto) { + await this.findOne(id); // throws if missing + return this.prisma.user.update({ where: { id }, data: dto }); + } + + async remove(id: string) { + await this.findOne(id); + await this.prisma.user.delete({ where: { id } }); + } +} +``` + +--- + +## Pipes & Validation + +Use `class-validator` + `class-transformer` with the global `ValidationPipe`. + +### Bootstrap validation globally + +```typescript +// main.ts +import { NestFactory } from '@nestjs/core'; +import { ValidationPipe } from '@nestjs/common'; +import { AppModule } from './app.module'; + +async function bootstrap() { + const app = await NestFactory.create(AppModule); + + app.useGlobalPipes(new ValidationPipe({ + whitelist: true, // Strip unknown properties + forbidNonWhitelisted: true, // Reject unknown properties with 400 + transform: true, // Auto-transform payloads to DTO instances + transformOptions: { enableImplicitConversion: true }, + })); + + await app.listen(3000); +} +bootstrap(); +``` + +### DTO with validation + +```typescript +// users/dto/create-user.dto.ts +import { IsEmail, IsString, MinLength, MaxLength, IsOptional, IsEnum } from 'class-validator'; + +export class CreateUserDto { + @IsEmail() + @MaxLength(254) + email: string; + + @IsString() + @MinLength(1) + @MaxLength(100) + name: string; + + @IsOptional() + @IsEnum(['admin', 'member', 'viewer']) + role?: string = 'member'; +} +``` + +```typescript +// users/dto/update-user.dto.ts +import { PartialType } from '@nestjs/mapped-types'; +import { CreateUserDto } from './create-user.dto'; + +export class UpdateUserDto extends PartialType(CreateUserDto) {} +``` + +`PartialType` makes all fields optional and preserves validators — no manual duplication. + +--- + +## Authentication + +### JWT Guard + +```typescript +// common/guards/jwt-auth.guard.ts +import { Injectable, CanActivate, ExecutionContext, UnauthorizedException } from '@nestjs/common'; +import { JwtService } from '@nestjs/jwt'; +import { Request } from 'express'; + +@Injectable() +export class JwtAuthGuard implements CanActivate { + constructor(private readonly jwtService: JwtService) {} + + async canActivate(context: ExecutionContext): Promise { + const request = context.switchToHttp().getRequest(); + const token = this.extractToken(request); + if (!token) throw new UnauthorizedException('Missing bearer token'); + + try { + const payload = await this.jwtService.verifyAsync(token); + request['user'] = payload; + } catch { + throw new UnauthorizedException('Invalid or expired token'); + } + return true; + } + + private extractToken(request: Request): string | undefined { + const [type, token] = request.headers.authorization?.split(' ') ?? []; + return type === 'Bearer' ? token : undefined; + } +} +``` + +### Apply guard globally with public route bypass + +```typescript +// app.module.ts +import { APP_GUARD } from '@nestjs/core'; +import { JwtAuthGuard } from './common/guards/jwt-auth.guard'; + +@Module({ + providers: [{ provide: APP_GUARD, useClass: JwtAuthGuard }], +}) +export class AppModule {} +``` + +```typescript +// common/decorators/public.decorator.ts +import { SetMetadata } from '@nestjs/common'; + +export const IS_PUBLIC_KEY = 'isPublic'; +export const Public = () => SetMetadata(IS_PUBLIC_KEY, true); +``` + +```typescript +// In JwtAuthGuard.canActivate(), add at the top: +const isPublic = this.reflector.getAllAndOverride(IS_PUBLIC_KEY, [ + context.getHandler(), + context.getClass(), +]); +if (isPublic) return true; +``` + +```typescript +// Usage — mark public routes +@Public() +@Get('health') +health() { return { status: 'ok' }; } +``` + +### Role-based access + +```typescript +// common/decorators/roles.decorator.ts +import { SetMetadata } from '@nestjs/common'; +export const ROLES_KEY = 'roles'; +export const Roles = (...roles: string[]) => SetMetadata(ROLES_KEY, roles); + +// common/guards/roles.guard.ts +@Injectable() +export class RolesGuard implements CanActivate { + constructor(private reflector: Reflector) {} + + canActivate(context: ExecutionContext): boolean { + const requiredRoles = this.reflector.getAllAndOverride(ROLES_KEY, [ + context.getHandler(), context.getClass(), + ]); + if (!requiredRoles) return true; + const { user } = context.switchToHttp().getRequest(); + return requiredRoles.includes(user.role); + } +} +``` + +--- + +## Exception Filters + +### Global Problem Details filter (RFC 9457) + +Consistent with the `openapi` skill's convention — all errors as `application/problem+json`. + +```typescript +// common/filters/problem-details.filter.ts +import { ExceptionFilter, Catch, ArgumentsHost, HttpException, HttpStatus } from '@nestjs/common'; +import { Response } from 'express'; + +@Catch() +export class ProblemDetailsFilter implements ExceptionFilter { + catch(exception: unknown, host: ArgumentsHost) { + const ctx = host.switchToHttp(); + const response = ctx.getResponse(); + + const status = exception instanceof HttpException + ? exception.getStatus() + : HttpStatus.INTERNAL_SERVER_ERROR; + + const exceptionResponse = exception instanceof HttpException + ? exception.getResponse() + : {}; + + const detail = typeof exceptionResponse === 'string' + ? exceptionResponse + : (exceptionResponse as any).message; + + response.status(status).json({ + type: `https://api.example.com/problems/${this.slugify(status)}`, + title: HttpStatus[status]?.replace(/_/g, ' ').toLowerCase() ?? 'Error', + status, + detail: Array.isArray(detail) ? detail.join('; ') : detail, + }); + } + + private slugify(status: number): string { + return (HttpStatus[status] ?? 'error').toLowerCase().replace(/_/g, '-'); + } +} +``` + +Register globally in `main.ts`: + +```typescript +app.useGlobalFilters(new ProblemDetailsFilter()); +``` + +--- + +## Database + +### Prisma integration (recommended) + +```typescript +// prisma/prisma.service.ts +import { Injectable, OnModuleInit, OnModuleDestroy } from '@nestjs/common'; +import { PrismaClient } from '@prisma/client'; + +@Injectable() +export class PrismaService extends PrismaClient implements OnModuleInit, OnModuleDestroy { + async onModuleInit() { await this.$connect(); } + async onModuleDestroy() { await this.$disconnect(); } +} +``` + +```typescript +// prisma/prisma.module.ts +import { Global, Module } from '@nestjs/common'; +import { PrismaService } from './prisma.service'; + +@Global() +@Module({ + providers: [PrismaService], + exports: [PrismaService], +}) +export class PrismaModule {} +``` + +Then inject `PrismaService` in any service: + +```typescript +@Injectable() +export class UsersService { + constructor(private readonly prisma: PrismaService) {} + // ... +} +``` + +### TypeORM alternative + +```typescript +// users/entities/user.entity.ts +import { Entity, PrimaryGeneratedColumn, Column, CreateDateColumn, UpdateDateColumn } from 'typeorm'; + +@Entity() +export class User { + @PrimaryGeneratedColumn('uuid') + id: string; + + @Column({ unique: true }) + email: string; + + @Column() + name: string; + + @Column({ default: 'member' }) + role: string; + + @CreateDateColumn() + createdAt: Date; + + @UpdateDateColumn() + updatedAt: Date; +} +``` + +--- + +## Testing + +### Unit testing a service + +```typescript +// users/users.service.spec.ts +import { Test, TestingModule } from '@nestjs/testing'; +import { UsersService } from './users.service'; +import { PrismaService } from '../prisma/prisma.service'; +import { NotFoundException } from '@nestjs/common'; + +describe('UsersService', () => { + let service: UsersService; + let prisma: PrismaService; + + beforeEach(async () => { + const module: TestingModule = await Test.createTestingModule({ + providers: [ + UsersService, + { + provide: PrismaService, + useValue: { + user: { + findUnique: jest.fn(), + create: jest.fn(), + update: jest.fn(), + delete: jest.fn(), + }, + }, + }, + ], + }).compile(); + + service = module.get(UsersService); + prisma = module.get(PrismaService); + }); + + it('throws NotFoundException when user does not exist', async () => { + jest.spyOn(prisma.user, 'findUnique').mockResolvedValue(null); + await expect(service.findOne('missing')).rejects.toThrow(NotFoundException); + }); +}); +``` + +### E2E testing a controller + +```typescript +// test/users.e2e-spec.ts +import { Test, TestingModule } from '@nestjs/testing'; +import { INestApplication, ValidationPipe } from '@nestjs/common'; +import * as request from 'supertest'; +import { AppModule } from '../src/app.module'; + +describe('UsersController (e2e)', () => { + let app: INestApplication; + + beforeAll(async () => { + const module: TestingModule = await Test.createTestingModule({ + imports: [AppModule], + }).compile(); + + app = module.createNestApplication(); + app.useGlobalPipes(new ValidationPipe({ whitelist: true, forbidNonWhitelisted: true })); + await app.init(); + }); + + afterAll(() => app.close()); + + it('POST /users — creates user', () => + request(app.getHttpServer()) + .post('/users') + .send({ email: 'test@example.com', name: 'Test' }) + .expect(201) + .expect((res) => { + expect(res.body).toHaveProperty('id'); + expect(res.body.email).toBe('test@example.com'); + })); + + it('POST /users — rejects invalid email', () => + request(app.getHttpServer()) + .post('/users') + .send({ email: 'not-an-email', name: 'Test' }) + .expect(400)); +}); +``` + +--- + +## OpenAPI + +NestJS has first-class OpenAPI generation via `@nestjs/swagger`. + +```typescript +// main.ts +import { SwaggerModule, DocumentBuilder } from '@nestjs/swagger'; + +const config = new DocumentBuilder() + .setTitle('Acme API') + .setVersion('1.0') + .addBearerAuth() + .build(); +const document = SwaggerModule.createDocument(app, config); +SwaggerModule.setup('docs', app, document); +``` + +Annotate DTOs for richer specs: + +```typescript +import { ApiProperty, ApiPropertyOptional } from '@nestjs/swagger'; + +export class CreateUserDto { + @ApiProperty({ example: 'jane@example.com', maxLength: 254 }) + @IsEmail() + email: string; + + @ApiPropertyOptional({ enum: ['admin', 'member', 'viewer'], default: 'member' }) + @IsOptional() + @IsEnum(['admin', 'member', 'viewer']) + role?: string = 'member'; +} +``` + +Use the `@nestjs/swagger` CLI plugin to auto-generate `@ApiProperty` decorators from TypeScript types — saves boilerplate. + +--- + +## Configuration + +Use `@nestjs/config` with Zod validation for type-safe env vars. + +```typescript +// config/env.validation.ts +import { z } from 'zod'; + +export const envSchema = z.object({ + NODE_ENV: z.enum(['development', 'production', 'test']).default('development'), + PORT: z.coerce.number().default(3000), + DATABASE_URL: z.string().url(), + JWT_SECRET: z.string().min(32), +}); + +export type Env = z.infer; +``` + +```typescript +// config/config.module.ts +import { ConfigModule } from '@nestjs/config'; +import { envSchema } from './env.validation'; + +export const AppConfigModule = ConfigModule.forRoot({ + validate: (config) => envSchema.parse(config), + isGlobal: true, +}); +``` + +--- + +## Interceptors + +### Request logging + +```typescript +// common/interceptors/logging.interceptor.ts +import { Injectable, NestInterceptor, ExecutionContext, CallHandler, Logger } from '@nestjs/common'; +import { Observable, tap } from 'rxjs'; + +@Injectable() +export class LoggingInterceptor implements NestInterceptor { + private readonly logger = new Logger('HTTP'); + + intercept(context: ExecutionContext, next: CallHandler): Observable { + const req = context.switchToHttp().getRequest(); + const { method, url } = req; + const start = Date.now(); + + return next.handle().pipe( + tap(() => { + const res = context.switchToHttp().getResponse(); + this.logger.log(`${method} ${url} ${res.statusCode} ${Date.now() - start}ms`); + }), + ); + } +} +``` + +### Response transform (envelope) + +```typescript +// common/interceptors/transform.interceptor.ts +import { Injectable, NestInterceptor, ExecutionContext, CallHandler } from '@nestjs/common'; +import { Observable, map } from 'rxjs'; + +@Injectable() +export class TransformInterceptor implements NestInterceptor { + intercept(_context: ExecutionContext, next: CallHandler): Observable<{ data: T }> { + return next.handle().pipe(map((data) => ({ data }))); + } +} +``` + +--- + +## Common Pitfalls + +1. **Circular dependencies.** Module A imports Module B which imports Module A. Use `forwardRef()` or restructure to break the cycle. If you need `forwardRef`, the architecture likely needs rethinking. +2. **Not using `whitelist: true` on ValidationPipe.** Without it, extra properties pass through silently — a security risk and a debugging nightmare. +3. **Overusing `@Global()` modules.** Only truly cross-cutting concerns (config, database, logging) should be global. Feature modules should explicitly import what they need. +4. **Testing with real database in unit tests.** Unit tests should mock the database layer. Use NestJS E2E tests (with `supertest`) for integration testing against a real DB. +5. **Forgetting to `await app.init()` in E2E tests.** Without it, providers aren't initialized and tests fail with cryptic DI errors. +6. **Putting business logic in controllers.** Controllers should only parse requests and return responses. All logic goes in services. +7. **Not using `PartialType` / `PickType` / `OmitType` for DTOs.** Duplicating validation rules across create/update DTOs leads to drift. Use mapped types. +8. **Ignoring graceful shutdown.** Call `app.enableShutdownHooks()` so `onModuleDestroy` lifecycle hooks fire on SIGTERM (critical for database connections in containers). + +--- + +## Related Skills + +- `openapi` — OpenAPI spec design (NestJS auto-generates from decorators) +- `typescript` — TypeScript patterns (NestJS is TypeScript-first) +- `postgresql` — Database design and query optimization +- `docker` — Containerizing NestJS apps +- `playwright` — E2E testing NestJS APIs through the browser +- `authentication` — JWT / OAuth2 patterns (framework-agnostic) +- `github-actions` — CI/CD for NestJS projects diff --git a/.claude/skills/background-jobs/SKILL.md b/.claude/skills/background-jobs/SKILL.md new file mode 100644 index 0000000..e29a610 --- /dev/null +++ b/.claude/skills/background-jobs/SKILL.md @@ -0,0 +1,333 @@ +--- +name: background-jobs +description: > + Use when implementing background task processing, job queues, or async work outside the request/response cycle. Trigger for keywords like Celery, BullMQ, Bull, task queue, background job, worker, cron job, scheduled task, async task, delayed job, or any mention of processing work outside the HTTP request lifecycle. Also activate when sending emails, generating reports, processing uploads, or handling webhooks asynchronously. +--- + +# Background Jobs + +## When to Use + +- Sending emails or notifications after an API response +- Processing file uploads (resize images, parse CSVs) +- Generating reports or exports +- Webhook delivery with retries +- Scheduled/cron tasks (cleanup, aggregation) +- Any work that would make an API response too slow (>500ms) + +## When NOT to Use + +- Simple in-request work under 200ms — just do it inline +- One-off scripts — use a CLI command instead +- Real-time bidirectional communication — use WebSockets or SSE + +--- + +## Python: Celery + +### Setup + +```python +# src/core/celery.py +from celery import Celery + +app = Celery('myapp') +app.config_from_object({ + 'broker_url': 'redis://localhost:6379/1', + 'result_backend': 'redis://localhost:6379/2', + 'task_serializer': 'json', + 'result_serializer': 'json', + 'accept_content': ['json'], + 'timezone': 'UTC', + 'task_track_started': True, + 'task_acks_late': True, # Re-deliver if worker crashes + 'worker_prefetch_multiplier': 1, # Fair scheduling +}) +``` + +### Define tasks + +```python +# src/tasks/email.py +from src.core.celery import app + +@app.task( + bind=True, + max_retries=3, + default_retry_delay=60, # 60s between retries +) +def send_welcome_email(self, user_id: str): + try: + user = get_user(user_id) + mailer.send(to=user.email, template='welcome', context={'name': user.name}) + except MailerError as exc: + self.retry(exc=exc) + +@app.task(bind=True, max_retries=5, default_retry_delay=300) +def process_upload(self, upload_id: str): + try: + upload = get_upload(upload_id) + result = parse_csv(upload.file_path) + save_results(upload_id, result) + except Exception as exc: + self.retry(exc=exc) +``` + +### Dispatch from FastAPI + +```python +from fastapi import APIRouter, status +from src.tasks.email import send_welcome_email + +router = APIRouter() + +@router.post("/api/users", status_code=status.HTTP_201_CREATED) +async def create_user(body: CreateUserRequest): + user = await save_user(body) + send_welcome_email.delay(str(user.id)) # Fire and forget + return user +``` + +### Scheduled tasks (Celery Beat) + +```python +# In celery config +app.conf.beat_schedule = { + 'cleanup-expired-sessions': { + 'task': 'src.tasks.cleanup.cleanup_sessions', + 'schedule': 3600.0, # Every hour + }, + 'daily-report': { + 'task': 'src.tasks.reports.generate_daily', + 'schedule': crontab(hour=2, minute=0), # 2:00 AM UTC + }, +} +``` + +### Run workers + +```bash +# Worker +celery -A src.core.celery worker --loglevel=info --concurrency=4 + +# Beat scheduler +celery -A src.core.celery beat --loglevel=info + +# Both (development only) +celery -A src.core.celery worker --beat --loglevel=info +``` + +--- + +## TypeScript: BullMQ + +### Setup + +```typescript +// src/core/queue.ts +import { Queue, Worker } from 'bullmq'; +import Redis from 'ioredis'; + +const connection = new Redis(process.env.REDIS_URL!, { maxRetriesPerRequest: null }); + +export function createQueue(name: string) { + return new Queue(name, { connection }); +} + +export function createWorker( + name: string, + processor: (job: { data: T }) => Promise, +) { + return new Worker(name, async (job) => processor(job), { + connection, + concurrency: 5, + }); +} +``` + +### Define queues and workers + +```typescript +// src/queues/email.queue.ts +import { createQueue, createWorker } from '../core/queue'; + +interface WelcomeEmailJob { + userId: string; + email: string; + name: string; +} + +export const emailQueue = createQueue('email'); + +export const emailWorker = createWorker('email', async (job) => { + await mailer.send({ + to: job.data.email, + template: 'welcome', + context: { name: job.data.name }, + }); +}); + +emailWorker.on('failed', (job, err) => { + console.error(`Email job ${job?.id} failed:`, err.message); +}); +``` + +### Dispatch from NestJS + +```typescript +// src/users/users.service.ts +import { Injectable } from '@nestjs/common'; +import { emailQueue } from '../queues/email.queue'; + +@Injectable() +export class UsersService { + async create(dto: CreateUserDto) { + const user = await this.prisma.user.create({ data: dto }); + + await emailQueue.add('welcome', { + userId: user.id, + email: user.email, + name: user.name, + }, { + attempts: 3, + backoff: { type: 'exponential', delay: 60_000 }, + }); + + return user; + } +} +``` + +### Scheduled/repeatable jobs + +```typescript +// Run every hour +await emailQueue.add('digest', { type: 'hourly' }, { + repeat: { every: 3600_000 }, +}); + +// Cron pattern — daily at 2 AM UTC +await emailQueue.add('daily-report', {}, { + repeat: { pattern: '0 2 * * *' }, +}); +``` + +### NestJS module integration + +```typescript +// src/queues/queues.module.ts +import { Module, OnModuleDestroy } from '@nestjs/common'; +import { emailQueue, emailWorker } from './email.queue'; + +@Module({ + providers: [ + { provide: 'EMAIL_QUEUE', useValue: emailQueue }, + ], + exports: ['EMAIL_QUEUE'], +}) +export class QueuesModule implements OnModuleDestroy { + async onModuleDestroy() { + await emailWorker.close(); + await emailQueue.close(); + } +} +``` + +--- + +## Job Design Patterns + +### Idempotent jobs + +Jobs may be retried. Design for idempotency: + +```python +# BAD — sends duplicate emails on retry +@app.task +def send_email(user_id): + send(user_id) + +# GOOD — check before sending +@app.task +def send_email(user_id): + if already_sent(user_id, 'welcome'): + return + send(user_id) + mark_sent(user_id, 'welcome') +``` + +### Small payloads + +```typescript +// BAD — large payload in queue +await queue.add('process', { csvData: '...10MB of CSV...' }); + +// GOOD — pass a reference +await queue.add('process', { uploadId: 'upload_123' }); +``` + +### Dead letter queues + +```typescript +// After max retries, move to DLQ for investigation +await emailQueue.add('welcome', data, { + attempts: 3, + backoff: { type: 'exponential', delay: 60_000 }, + removeOnComplete: true, + removeOnFail: false, // Keep failed jobs for inspection +}); +``` + +--- + +## Testing + +### Python (Celery) + +```python +# Always run tasks eagerly in tests +@pytest.fixture(autouse=True) +def celery_eager(settings): + settings.CELERY_TASK_ALWAYS_EAGER = True + settings.CELERY_TASK_EAGER_PROPAGATES = True + +def test_welcome_email_sent(mock_mailer): + send_welcome_email("user_123") + mock_mailer.send.assert_called_once() +``` + +### TypeScript (BullMQ) + +```typescript +describe('email worker', () => { + it('should send welcome email', async () => { + const sendSpy = vi.spyOn(mailer, 'send'); + + // Process job directly (skip queue) + await emailWorker.run({ data: { userId: '1', email: 'a@b.com', name: 'Test' } }); + + expect(sendSpy).toHaveBeenCalledWith(expect.objectContaining({ to: 'a@b.com' })); + }); +}); +``` + +--- + +## Common Pitfalls + +1. **Non-idempotent tasks.** Retries will duplicate side effects. Always check before acting. +2. **Large payloads in queue.** Store data in DB/S3, pass only IDs through the queue. +3. **No retry limits.** Always set `max_retries` / `attempts`. Infinite retries waste resources. +4. **Missing dead letter handling.** Failed jobs need investigation. Don't silently discard them. +5. **Blocking the event loop (Node.js).** CPU-heavy work in BullMQ workers blocks other jobs. Use `worker threads` or separate processes. +6. **Not monitoring queue depth.** Queue buildup indicates workers can't keep up. Alert on queue size. + +--- + +## Related Skills + +- `redis` — Redis as the message broker for both Celery and BullMQ +- `docker` — Running workers as separate containers +- `fastapi` — Dispatching Celery tasks from FastAPI endpoints +- `nestjs` — BullMQ integration with NestJS modules +- `logging` — Structured logging for job execution tracking diff --git a/.claude/skills/methodology/brainstorming/SKILL.md b/.claude/skills/brainstorming/SKILL.md similarity index 61% rename from .claude/skills/methodology/brainstorming/SKILL.md rename to .claude/skills/brainstorming/SKILL.md index cb191d4..b844e59 100644 --- a/.claude/skills/methodology/brainstorming/SKILL.md +++ b/.claude/skills/brainstorming/SKILL.md @@ -1,7 +1,7 @@ --- name: brainstorming description: > - Trigger this skill whenever the user wants to design, explore, or ideate on ANY new feature, architecture decision, or unclear requirement. Activate for keywords like "brainstorm", "design", "explore", "what if", "how should we", "options for", "trade-offs", or any open-ended question about implementation approach. Also trigger when requirements are vague, ambiguous, or when multiple valid solutions exist -- err on the side of brainstorming before jumping into code. + Use when the user wants to design, explore, or ideate on ANY new feature, architecture decision, or unclear requirement. Activate for keywords like "brainstorm", "design", "explore", "what if", "how should we", "options for", "trade-offs", or any open-ended question about implementation approach. Also trigger when requirements are vague, ambiguous, or when multiple valid solutions exist -- err on the side of brainstorming before jumping into code. --- # Brainstorming @@ -191,7 +191,65 @@ For informed technology choices: --- +## Stack-Specific Brainstorming Examples + +These show what Phase 2 (Exploration) output looks like for different domains: + +### FastAPI endpoint design + +```markdown +## Approach 1: REST + JWT Bearer Auth (Recommended) +POST /api/orders with Pydantic v2 validation, async SQLAlchemy. +- Pros: Simple, cacheable, great OpenAPI docs via FastAPI +- Cons: Multiple round-trips for nested resources + +## Approach 2: GraphQL + API Key Auth +Single /graphql endpoint with Strawberry, API key in header. +- Pros: Flexible queries, single round-trip for nested data +- Cons: Caching harder, team unfamiliar with Strawberry + +**Decision**: REST — team knows it, OpenAPI auto-docs save time, +nested resources not needed for this feature. +``` + +### React data table component + +```markdown +## Approach 1: TanStack Table + URL Params (Recommended) +Server component fetches data, client component for interactions. +Sort/filter state in URL search params (shareable links). +- Pros: Bookmarkable state, SSR-friendly, no global store needed +- Cons: URL parsing boilerplate + +## Approach 2: Zustand Store + SWR +Client-only with SWR for fetching, Zustand for table state. +- Pros: Simple state management, familiar pattern +- Cons: Not SSR-friendly, state lost on refresh + +**Decision**: TanStack Table + URL params — users need to share +filtered views, and it works with Next.js App Router. +``` + +### Database multi-tenancy + +```markdown +## Approach 1: Shared Table + tenant_id + RLS (Recommended) +Single `orders` table with `tenant_id` column, PostgreSQL RLS policies. +- Pros: Simple migrations, single connection pool, no schema sprawl +- Cons: Must never forget WHERE tenant_id = ? (RLS prevents this) + +## Approach 2: Schema-per-tenant +Each tenant gets own PostgreSQL schema, selected via search_path. +- Pros: Strong isolation, easy per-tenant backup/restore +- Cons: Migration complexity grows linearly with tenants + +**Decision**: Shared table + RLS — we have <100 tenants, RLS gives +isolation guarantees without migration pain. +``` + +--- + ## Related Skills -- `methodology/writing-plans` -- After brainstorming produces a validated design, use writing-plans to create a detailed implementation plan -- `methodology/sequential-thinking` -- For complex problems that benefit from structured step-by-step reasoning during the brainstorming process +- `writing-plans` -- After brainstorming produces a validated design, use writing-plans to create a detailed implementation plan +- `sequential-thinking` -- For complex problems that benefit from structured step-by-step reasoning during the brainstorming process diff --git a/.claude/skills/methodology/brainstorming/references/question-patterns.md b/.claude/skills/brainstorming/references/question-patterns.md similarity index 100% rename from .claude/skills/methodology/brainstorming/references/question-patterns.md rename to .claude/skills/brainstorming/references/question-patterns.md diff --git a/.claude/skills/caching/SKILL.md b/.claude/skills/caching/SKILL.md new file mode 100644 index 0000000..27fc74f --- /dev/null +++ b/.claude/skills/caching/SKILL.md @@ -0,0 +1,60 @@ +--- +name: caching +description: > + Use when implementing memoization, HTTP cache headers, Redis caching, CDN configuration, or in-memory caches. Also activate whenever code deals with Cache-Control headers, ETags, functools.lru_cache, React useMemo, TanStack Query cache, or any caching strategy. Applies to cache invalidation, TTL policies, and cache-aside patterns. +--- + +# Caching + +## When to Use + +- Memoizing expensive function calls (lru_cache, useMemo, node-cache) +- Setting HTTP cache headers (Cache-Control, ETag, Last-Modified) +- Implementing Redis cache-aside pattern for database query results +- Configuring CDN caching for static assets and API responses +- Building multi-layer caches (in-memory + Redis + CDN) +- Implementing cache invalidation strategies + +## When NOT to Use + +- Data that changes on every request (real-time prices, live feeds) +- Security-sensitive responses that must never be cached (auth tokens, personal data) +- Development environments where stale data causes confusion + +--- + +## Quick Reference + +| Topic | Reference | Key content | +|-------|-----------|-------------| +| All caching patterns | `references/patterns.md` | Memoization, HTTP headers, ETags, Redis, CDN, multi-layer, invalidation | +| Decision tree | `references/caching-decision-tree.md` | When to use which caching strategy | + +--- + +## Best Practices + +1. **Cache at the right layer.** In-memory for hot paths (<1ms), Redis for shared state (<5ms), CDN for static/semi-static content. +2. **Always set TTLs.** Every cache entry must expire. Unbounded caches grow until they crash. +3. **Use cache-aside (lazy loading) by default.** Read from cache, miss goes to DB, write result to cache. Simplest and most predictable pattern. +4. **Invalidate on write.** When data changes, delete the cache key immediately. Don't wait for TTL expiry. +5. **Use ETag-based validation** for HTTP caching. Cheaper than full responses and guarantees freshness. +6. **Prevent cache stampede.** When a popular key expires, use distributed locks or stale-while-revalidate to prevent all requests from hitting the DB simultaneously. +7. **Monitor cache hit rates.** A cache with <80% hit rate may not be worth the complexity. Measure before optimizing. + +## Common Pitfalls + +1. **Caching without TTL** — memory grows unboundedly until OOM. +2. **Cache invalidation bugs** — stale data served after writes. Always invalidate on mutation. +3. **Caching user-specific data with shared keys** — one user sees another's data. +4. **Over-caching in development** — confusing stale responses with bugs. +5. **Ignoring serialization costs** — caching large objects in Redis costs more in ser/deser than the DB query saved. +6. **Not handling cache failures gracefully** — if Redis is down, fall through to DB, don't crash. + +--- + +## Related Skills + +- `databases` — Redis patterns and database query optimization +- `backend-frameworks` — Framework-specific cache middleware +- `frontend` — React useMemo, TanStack Query cache diff --git a/.claude/skills/patterns/caching/references/caching-decision-tree.md b/.claude/skills/caching/references/caching-decision-tree.md similarity index 100% rename from .claude/skills/patterns/caching/references/caching-decision-tree.md rename to .claude/skills/caching/references/caching-decision-tree.md diff --git a/.claude/skills/patterns/caching/SKILL.md b/.claude/skills/caching/references/patterns.md similarity index 96% rename from .claude/skills/patterns/caching/SKILL.md rename to .claude/skills/caching/references/patterns.md index 8d0d8a6..cfdbb9a 100644 --- a/.claude/skills/patterns/caching/SKILL.md +++ b/.claude/skills/caching/references/patterns.md @@ -1,8 +1,5 @@ ---- -name: caching -description: > - Caching patterns for web applications, APIs, and data layers. Use this skill when implementing memoization, HTTP cache headers, Redis caching, CDN configuration, or in-memory caches. Trigger whenever code deals with Cache-Control headers, ETags, functools.lru_cache, React useMemo, TanStack Query cache, or any caching strategy. Also applies to cache invalidation, TTL policies, and cache-aside patterns. ---- +# Caching — Patterns + # Caching @@ -779,8 +776,8 @@ export default async function ProductList({ categoryId }: Props) { ## Related Skills -- `patterns/state-management` — Client-side state management patterns that interact with cache layers -- `databases/postgresql` — Query optimization and connection pooling that complement caching strategies -- `databases/mongodb` — MongoDB query patterns and when to add a cache layer -- `frameworks/nextjs` — Next.js data fetching, ISR, and caching architecture -- `patterns/api-client` — Client-side caching for API responses +- `state-management` — Client-side state management patterns that interact with cache layers +- `postgresql` — Query optimization and connection pooling that complement caching strategies +- `mongodb` — MongoDB query patterns and when to add a cache layer +- `nextjs` — Next.js data fetching, ISR, and caching architecture +- `api-client` — Client-side caching for API responses diff --git a/.claude/skills/condition-based-waiting/SKILL.md b/.claude/skills/condition-based-waiting/SKILL.md new file mode 100644 index 0000000..56cd7b2 --- /dev/null +++ b/.claude/skills/condition-based-waiting/SKILL.md @@ -0,0 +1,208 @@ +--- +name: condition-based-waiting +description: > + Use when waiting on external conditions like CI pipeline runs, deployments, long builds, database migrations, or test suites. Trigger for keywords like "wait for", "check status", "poll", "monitor", "is it done", "build running", "deploy in progress", or when a background process needs to complete before the next step. Also activate when using run_in_background or Monitor tools in Claude Code. +--- + +# Condition-Based Waiting + +## When to Use + +- CI/CD pipeline is running and you need results before proceeding +- Deployment is in progress and you need to verify it succeeded +- Long-running build (Next.js, Docker) is executing +- Database migration is applying +- Test suite takes more than 30 seconds + +## When NOT to Use + +- Commands that complete in under 10 seconds (just run them normally) +- Checking static state that won't change (read the file instead) +- Polling for human action (ask the user instead) + +--- + +## Claude Code Patterns + +### Background execution for long commands + +Use `run_in_background` when a command takes more than ~30 seconds: + +```bash +# Long test suite — run in background, get notified when done +pytest -v --cov=src # run_in_background: true + +# Docker build +docker build -t myapp . # run_in_background: true + +# Next.js production build +next build # run_in_background: true + +# NestJS build + test +npm run build && npm test # run_in_background: true +``` + +You'll be notified automatically when the command completes — **do not poll or sleep**. + +### Monitor tool for streaming output + +Use Monitor when you need to watch for specific output patterns: + +```bash +# Watch for build completion +until curl -sf http://localhost:3000/health; do sleep 2; done + +# Watch for migration completion +until alembic check 2>&1 | grep -q "No new upgrade"; do sleep 5; done +``` + +--- + +## Checking CI/CD Status + +### GitHub Actions + +```bash +# Watch a running workflow (blocks until complete) +gh run watch + +# Check status of the latest run +gh run view --json status,conclusion + +# Check specific workflow +gh run list --workflow=ci.yml --limit=1 --json status,conclusion + +# Wait for all checks on a PR +gh pr checks --watch +``` + +### After CI completes + +```bash +# Get detailed results +gh run view --log-failed + +# Re-run failed jobs only +gh run rerun --failed +``` + +--- + +## Checking Deployments + +### Health check polling + +```bash +# Wait for deployment to be healthy +until curl -sf https://staging.example.com/health | grep -q '"status":"ok"'; do + sleep 5 +done +echo "Deployment is healthy" +``` + +### Vercel / Cloudflare + +```bash +# Vercel — check latest deployment status +npx vercel ls --limit=1 + +# Cloudflare Pages — check deployment +npx wrangler pages deployment list --project-name=myapp +``` + +--- + +## Checking Build Output + +### Framework-specific patterns + +```bash +# Next.js — watch for "Compiled successfully" +# (use run_in_background for `next build`, read output when notified) + +# Python — watch for test results +pytest -v --tb=short # run_in_background: true + +# Docker — watch for "Successfully built" +docker build -t myapp . # run_in_background: true +``` + +### Database migrations + +```bash +# Alembic (Python) +alembic upgrade head # run_in_background: true for large migrations + +# Prisma (TypeScript) +npx prisma migrate deploy # run_in_background: true + +# Verify migration status +alembic check # Python +npx prisma migrate status # TypeScript +``` + +--- + +## Anti-Patterns + +### Don't: Sleep loops + +```bash +# BAD — burns cache, wastes tokens +sleep 60 && check_status +sleep 60 && check_status +sleep 60 && check_status + +# GOOD — use run_in_background or until-loop with Monitor +``` + +### Don't: Poll too frequently + +```bash +# BAD — checking every second +while true; do curl localhost:3000/health; sleep 1; done + +# GOOD — reasonable interval based on expected duration +until curl -sf localhost:3000/health; do sleep 5; done +``` + +### Don't: Wait without timeouts + +```bash +# BAD — waits forever +until curl -sf localhost:3000/health; do sleep 5; done + +# GOOD — timeout after 5 minutes +timeout 300 bash -c 'until curl -sf localhost:3000/health; do sleep 5; done' +``` + +### Don't: Guess completion + +```markdown +BAD: "The build probably finished by now, let's proceed" +GOOD: "Let me check the build status before proceeding" +``` + +--- + +## Timing Guide + +| Operation | Expected Duration | Check Interval | Approach | +|-----------|------------------|----------------|----------| +| Unit tests (small) | 5-30s | N/A | Run inline | +| Unit tests (large) | 30s-5m | N/A | `run_in_background` | +| `next build` | 30s-3m | N/A | `run_in_background` | +| Docker build | 1-10m | N/A | `run_in_background` | +| CI pipeline | 2-15m | 30s | `gh run watch` | +| Deployment | 1-10m | 5s | Health check poll | +| DB migration (small) | 5-30s | N/A | Run inline | +| DB migration (large) | 1-30m | N/A | `run_in_background` | + +--- + +## Related Skills + +- `verification-before-completion` — After waiting, verify the result before claiming success +- `github-actions` — CI/CD workflow patterns +- `docker` — Container build patterns +- `systematic-debugging` — When the thing you're waiting for fails diff --git a/.claude/skills/databases/SKILL.md b/.claude/skills/databases/SKILL.md new file mode 100644 index 0000000..ed7e689 --- /dev/null +++ b/.claude/skills/databases/SKILL.md @@ -0,0 +1,64 @@ +--- +name: databases +description: > + Use when working with PostgreSQL, MongoDB, or Redis — including schema design, queries, indexing, migrations, connection pooling, caching layers, or any database operation. Also activate for keywords like SQL, aggregation pipeline, BSON, ioredis, alembic, prisma migrate, django migrate, EXPLAIN ANALYZE, ORM configuration, or NoSQL data modeling. +--- + +# Databases + +## When to Use + +- PostgreSQL database operations, SQL query optimization, schema design +- JSONB document storage, full-text search, window functions, CTEs +- MongoDB document modeling, aggregation pipelines, semi-structured data +- Redis caching, session storage, rate limiting, pub/sub, job queues, distributed locks +- Database migrations — adding/modifying tables, columns, indexes, constraints +- Resolving migration conflicts, rolling back failed migrations + +## When NOT to Use + +- Simple key-value caching within a single process — use `functools.lru_cache` or `Map` +- File-based storage that doesn't need a database engine +- Static data or configuration that belongs in environment variables + +--- + +## Quick Reference + +| Topic | Reference | Key tools | +|-------|-----------|-----------| +| PostgreSQL | `references/postgresql.md` | SQL, SQLAlchemy, Prisma, EXPLAIN ANALYZE, pg_stat_statements | +| MongoDB | `references/mongodb.md` | Aggregation, Mongoose, Motor, document schemas, ESR indexing | +| Redis | `references/redis.md` | Caching, pub/sub, ioredis, BullMQ, session storage, distributed locks | +| Migrations | `references/migrations.md` | Alembic, Prisma Migrate, Django migrations, rollback strategies | + +--- + +## Best Practices + +1. **Use parameterized queries everywhere.** Never concatenate user input into SQL strings. +2. **Design schema around access patterns.** Ask "how will I read this?" before "how does this relate?" Embed data fetched together (MongoDB); normalize data accessed independently (PostgreSQL). +3. **Index foreign keys and query fields.** PostgreSQL doesn't auto-index FK child columns. MongoDB queries without indexes trigger full collection scans. +4. **Use appropriate consistency levels.** `TIMESTAMPTZ` over `TIMESTAMP` (PostgreSQL). `w: "majority"` for durable writes (MongoDB). TTLs on every Redis cache key. +5. **Monitor query performance.** `pg_stat_statements` (PostgreSQL), `db.setProfilingLevel(1)` (MongoDB), connection pool metrics (all). +6. **Use bulk/batch operations.** `bulkWrite` (MongoDB), `COPY` (PostgreSQL), pipelines (Redis) for high-throughput writes. +7. **Never edit deployed migrations.** Create a new migration instead of modifying one already applied. +8. **Test rollback paths.** Always verify your downgrade/rollback strategy before deploying schema changes. + +## Common Pitfalls + +1. **N+1 queries from ORM lazy loading.** Use eager loading (`joinedload`, `select_related`, `$lookup` with caution). +2. **Table locks during migrations.** Use `CREATE INDEX CONCURRENTLY` (PostgreSQL). Batch backfills for large tables. +3. **Unbounded growth.** Dead tuples from UPDATE-heavy workloads (PostgreSQL). Arrays exceeding 16MB document limit (MongoDB). Redis keys without TTLs. +4. **OFFSET pagination on large datasets.** Use keyset/cursor pagination instead. +5. **Connection exhaustion.** Use connection pools (PgBouncer, application-level pools). Never open per-request connections. +6. **Cache stampede.** When a popular Redis key expires, many requests hit the DB simultaneously. Use distributed locks or stale-while-revalidate. +7. **Running `migrate reset` in production.** This drops all data. + +--- + +## Related Skills + +- `backend-frameworks` — Framework-specific ORM integration +- `error-handling` — Database error handling patterns +- `logging` — Query logging and slow query detection diff --git a/.claude/skills/databases/mongodb/references/schema-patterns.md b/.claude/skills/databases/mongodb/references/schema-patterns.md deleted file mode 100644 index c488607..0000000 --- a/.claude/skills/databases/mongodb/references/schema-patterns.md +++ /dev/null @@ -1,237 +0,0 @@ -# MongoDB Schema Design Patterns - -Quick reference for embedding vs referencing decisions and common schema patterns. - -## Embedding vs Referencing Decision Tree - -``` -What is the relationship cardinality? -| -+-- One-to-Few (< 50 items)? -| --> EMBED in parent document -| Example: user.addresses, post.tags -| -+-- One-to-Many (50 - 1000s)? -| | -| +-- Child data always accessed with parent? -| | --> EMBED (but watch 16 MB doc limit) -| | -| +-- Child data accessed independently? -| | --> REFERENCE (store child _id in parent array) -| | -| +-- Need atomic updates across parent + children? -| --> EMBED -| -+-- One-to-Millions? -| --> REFERENCE from child to parent -| Example: log_entry.host_id (not host.log_entry_ids) -| -+-- Many-to-Many? - --> REFERENCE with array of _ids on one or both sides - Example: student.course_ids[], course.student_ids[] -``` - -## Decision Factors - -| Factor | Favor Embedding | Favor Referencing | -|--------|----------------|-------------------| -| **Read pattern** | Always read together | Read independently | -| **Write pattern** | Infrequent child updates | Frequent child updates | -| **Data size** | Small, bounded children | Large or growing children | -| **Atomicity** | Need single-doc transactions | Can tolerate multi-doc txn | -| **Duplication** | OK to denormalize | Must avoid duplication | -| **Cardinality** | Few items | Many/unbounded items | -| **Document size** | Well under 16 MB limit | Approaching 16 MB | - -## Pattern Catalog - -### 1. Subset Pattern - -**Problem**: Document is large but reads only need a few fields from embedded data. - -**Solution**: Embed a subset; keep full data in a separate collection. - -```javascript -// products collection - fast reads for listing pages -{ - _id: ObjectId("..."), - name: "Widget", - price: 29.99, - // Only the 10 most recent reviews (subset) - recent_reviews: [ - { user: "alice", rating: 5, text: "Great!", date: ISODate("...") } - ], - review_count: 247 -} - -// reviews collection - full review data -{ - _id: ObjectId("..."), - product_id: ObjectId("..."), - user: "alice", - rating: 5, - text: "Great!", - date: ISODate("..."), - helpful_votes: 12 -} -``` - -**When to use**: Product pages, user profiles, any "preview + detail" pattern. - -### 2. Computed Pattern - -**Problem**: Expensive aggregation queries run repeatedly on the same data. - -**Solution**: Pre-compute and store the result, update on write. - -```javascript -// movies collection -{ - _id: ObjectId("..."), - title: "Example Movie", - // Pre-computed from screenings collection - computed: { - total_revenue: 1250000, - avg_rating: 4.2, - rating_count: 843, - last_computed: ISODate("2025-01-15T00:00:00Z") - } -} -``` - -**Update strategy**: On each new rating, increment count and recalculate average. Or use a background job for less time-sensitive data. - -**When to use**: Dashboards, leaderboards, summary statistics. - -### 3. Bucket Pattern - -**Problem**: Many small, time-series documents create overhead (indexes, storage per doc). - -**Solution**: Group related data into fixed-size buckets. - -```javascript -// sensor_readings collection - one doc per sensor per hour -{ - sensor_id: "sensor-42", - bucket_start: ISODate("2025-01-15T14:00:00Z"), - bucket_end: ISODate("2025-01-15T14:59:59Z"), - count: 60, - readings: [ - { ts: ISODate("2025-01-15T14:00:00Z"), temp: 22.1, humidity: 45 }, - { ts: ISODate("2025-01-15T14:01:00Z"), temp: 22.3, humidity: 44 } - // ... up to 60 readings per bucket - ], - // Pre-computed aggregates for the bucket - summary: { - avg_temp: 22.4, - min_temp: 21.8, - max_temp: 23.1 - } -} -``` - -**Bucket sizing**: Choose a size that balances doc count reduction vs update frequency. Common choices: 1 hour, 1 day, 100 events. - -**When to use**: IoT, time-series, event logging, analytics. - -### 4. Outlier Pattern - -**Problem**: A few documents have vastly more data than the norm (e.g., a viral post with millions of likes). - -**Solution**: Flag outliers and overflow into separate documents. - -```javascript -// books collection - normal case -{ - _id: ObjectId("..."), - title: "Normal Book", - customers_purchased: ["user1", "user2", "user3"], - has_overflow: false -} - -// books collection - outlier (bestseller) -{ - _id: ObjectId("..."), - title: "Bestseller", - customers_purchased: ["user1", "user2", /* ... first 1000 */], - has_overflow: true -} - -// book_purchases_overflow collection -{ - book_id: ObjectId("..."), - page: 2, - customers_purchased: ["user1001", "user1002", /* ... next 1000 */] -} -``` - -**When to use**: Social media (viral posts), e-commerce (bestsellers), any data with power-law distribution. - -### 5. Extended Reference Pattern - -**Problem**: Frequent joins (lookups) to get a few fields from a referenced document. - -**Solution**: Copy the most-accessed fields into the referencing document. - -```javascript -// orders collection -{ - _id: ObjectId("..."), - date: ISODate("..."), - customer_id: ObjectId("..."), - // Extended reference - copied fields for fast reads - customer_name: "Alice Smith", - customer_email: "alice@example.com", - items: [ - { - product_id: ObjectId("..."), - product_name: "Widget", // copied - price: 29.99, // copied (snapshot at time of order) - quantity: 2 - } - ] -} -``` - -**Trade-off**: Stale data is acceptable (order snapshots price at purchase time). For data that must be current, keep only the reference. - -**When to use**: Orders (snapshot pricing), notifications (snapshot user name), audit logs. - -### 6. Polymorphic Pattern - -**Problem**: Objects share some fields but differ in others (e.g., different product types). - -**Solution**: Store in a single collection with a type discriminator. - -```javascript -// vehicles collection -{ type: "car", make: "Toyota", doors: 4, trunk_size_liters: 450 } -{ type: "truck", make: "Ford", doors: 2, payload_kg: 5000 } -{ type: "motorcycle", make: "Harley", engine_cc: 1200 } -``` - -**Index strategy**: Index common fields. Use partial indexes for type-specific fields. - -```javascript -db.vehicles.createIndex( - { payload_kg: 1 }, - { partialFilterExpression: { type: "truck" } } -); -``` - -**When to use**: Product catalogs, content management (articles, videos, images), mixed event streams. - -## Anti-Patterns - -| Mistake | Problem | Fix | -|---------|---------|-----| -| Unbounded array growth | Document exceeds 16 MB | Use bucket or outlier pattern | -| Deep nesting (> 3 levels) | Hard to query and index | Flatten or reference | -| Normalizing everything | Too many lookups, slow reads | Embed when read together | -| Embedding large blobs | Wastes RAM in working set | Store in GridFS or S3 | -| No schema validation | Inconsistent data over time | Use JSON Schema validation | -| Indexing every field | Slow writes, wasted space | Index based on query patterns | - -## Schema Validation - -Use `db.createCollection()` with `$jsonSchema` validator to enforce structure. Set `validationLevel: "moderate"` to apply only on inserts and updates (not existing docs). diff --git a/.claude/skills/databases/postgresql/references/index-decision-tree.md b/.claude/skills/databases/postgresql/references/index-decision-tree.md deleted file mode 100644 index 4f92dc4..0000000 --- a/.claude/skills/databases/postgresql/references/index-decision-tree.md +++ /dev/null @@ -1,173 +0,0 @@ -# PostgreSQL Index Decision Tree - -Quick reference for choosing the right index type. - -## Decision Tree - -``` -What are you querying? -| -+-- Equality (=) or Range (<, >, BETWEEN, ORDER BY)? -| | -| +-- On a single scalar column? -| | --> B-tree (default) -| | -| +-- On a timestamp/date column with append-only inserts? -| | --> BRIN (much smaller than B-tree) -| | -| +-- Need the index to also return columns without table lookup? -| --> Covering Index (B-tree with INCLUDE) -| -+-- Array containment (@>, &&) or JSONB queries? -| --> GIN -| -+-- Full-text search (tsvector, @@)? -| --> GIN -| -+-- Geometric/spatial data (points, polygons, PostGIS)? -| --> GiST -| -+-- Range types (int4range, tsrange, overlaps)? -| --> GiST -| -+-- Nearest-neighbor / distance queries (KNN)? -| --> GiST (or SP-GiST for partitioned space) -| -+-- Only a subset of rows match your WHERE clause? -| --> Partial Index (any type + WHERE filter) -| -+-- Trigram similarity (LIKE '%pattern%', pg_trgm)? -| --> GIN with pg_trgm (or GiST for smaller, slower) -| -+-- Hash equality only (= but never range)? - --> Hash index (rarely better than B-tree in practice) -``` - -## Index Type Comparison - -| Type | Best For | Operators | Size | Write Cost | Notes | -|------|----------|-----------|------|------------|-------| -| **B-tree** | Equality, range, sorting | `= < > <= >= BETWEEN IN IS NULL` | Medium | Low | Default. Covers 90% of cases. | -| **GIN** | Multi-valued data | `@> && @@ ? ?& ?|` | Large | High (slow updates) | Best for arrays, JSONB, full-text. Use `fastupdate=on`. | -| **GiST** | Spatial, ranges, nearest-neighbor | `<< >> && @> <@ <->` | Medium | Medium | Lossy for some types. Supports KNN. | -| **SP-GiST** | Partitioned search spaces | Same as GiST | Medium | Medium | Good for phone numbers, IP addresses, non-balanced trees. | -| **BRIN** | Large sequential/append-only tables | `= < > <= >=` | Tiny | Very Low | 1000x smaller than B-tree. Only effective when physical order correlates with column values. | -| **Hash** | Equality only | `=` | Medium | Low | WAL-logged since PG10. Rarely outperforms B-tree. | - -## Common Patterns - -### Covering Index (Index-Only Scans) - -Avoid heap lookups by including extra columns: - -```sql --- Query: SELECT email, name FROM users WHERE email = ? -CREATE INDEX idx_users_email_covering - ON users (email) INCLUDE (name); -``` - -### Partial Index (Filtered) - -Index only the rows you actually query: - -```sql --- Only index active orders (skip 95% of rows) -CREATE INDEX idx_orders_active - ON orders (created_at) - WHERE status = 'active'; -``` - -### Composite Index (Multi-Column) - -Column order matters -- put equality columns first, range columns last: - -```sql --- Query: WHERE tenant_id = ? AND created_at > ? -CREATE INDEX idx_events_tenant_date - ON events (tenant_id, created_at); -``` - -### Expression Index - -Index a computed value: - -```sql -CREATE INDEX idx_users_lower_email - ON users (lower(email)); -``` - -### GIN for JSONB - -```sql --- Index all keys and values in a JSONB column -CREATE INDEX idx_metadata_gin - ON products USING gin (metadata jsonb_path_ops); - --- Supports: metadata @> '{"color": "red"}' -``` - -### GiST for Range Overlap - -```sql -CREATE INDEX idx_reservations_during - ON reservations USING gist (during); - --- Supports: WHERE during && '[2025-01-01, 2025-01-31]'::daterange -``` - -### BRIN for Time-Series - -```sql --- Table has millions of rows inserted in timestamp order -CREATE INDEX idx_logs_ts_brin - ON logs USING brin (created_at) - WITH (pages_per_range = 32); -``` - -## Sizing Rules of Thumb - -| Table Rows | B-tree Size | BRIN Size | GIN Size | -|------------|-------------|-----------|----------| -| 1M | ~20 MB | ~50 KB | ~30 MB | -| 10M | ~200 MB | ~500 KB | ~300 MB | -| 100M | ~2 GB | ~5 MB | ~3 GB | - -## Diagnostic Queries - -```sql --- Check if an index is being used -EXPLAIN (ANALYZE, BUFFERS) SELECT ...; - --- Find unused indexes -SELECT indexrelname, idx_scan -FROM pg_stat_user_indexes -WHERE idx_scan = 0 -ORDER BY pg_relation_size(indexrelid) DESC; - --- Check index size -SELECT pg_size_pretty(pg_relation_size('idx_name')); - --- Index bloat estimate -SELECT * FROM pgstatindex('idx_name'); -``` - -## Anti-Patterns - -| Mistake | Why It Hurts | -|---------|-------------| -| Indexing every column | Slows writes, wastes disk, confuses planner | -| Wrong column order in composite | Index cannot be used for the query | -| GIN on tiny tables | Overhead exceeds benefit | -| B-tree on low-cardinality columns | Planner prefers seq scan anyway | -| Missing `CONCURRENTLY` on production | Locks the table during index build | -| Forgetting `ANALYZE` after bulk load | Planner uses stale statistics | - -## Safe Index Creation - -```sql --- Non-blocking index creation (no table lock) -CREATE INDEX CONCURRENTLY idx_name ON table (column); - --- Always run ANALYZE after bulk operations -ANALYZE table; -``` diff --git a/.claude/skills/databases/postgresql/templates/migration-template.sql b/.claude/skills/databases/postgresql/templates/migration-template.sql deleted file mode 100644 index 8e9a875..0000000 --- a/.claude/skills/databases/postgresql/templates/migration-template.sql +++ /dev/null @@ -1,143 +0,0 @@ --- ============================================================================= --- Migration: [DESCRIPTION] --- Created: [DATE] --- Author: [AUTHOR] --- Ticket: [TICKET-ID] --- ============================================================================= --- --- SAFETY CHECKLIST (review before running): --- [ ] Tested on staging with production-size data --- [ ] Backward compatible with current application code --- [ ] No exclusive locks on large tables during peak hours --- [ ] Rollback (DOWN) section tested independently --- [ ] Estimated run time: ___ --- [ ] Estimated lock duration: ___ --- - --- ============================================================ --- UP MIGRATION --- ============================================================ - -BEGIN; - --- Set a statement timeout to prevent long-running locks. --- Adjust as needed; remove for data-only migrations. -SET LOCAL lock_timeout = '5s'; -SET LOCAL statement_timeout = '30s'; - --- ------------------------------------ --- 1. Schema changes --- ------------------------------------ - --- Add new table --- CREATE TABLE IF NOT EXISTS example ( --- id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, --- name text NOT NULL, --- created_at timestamptz NOT NULL DEFAULT now(), --- updated_at timestamptz NOT NULL DEFAULT now() --- ); - --- Add column (safe: does not rewrite table) --- ALTER TABLE example ADD COLUMN IF NOT EXISTS description text; - --- Add column with default (PG 11+: does not rewrite table) --- ALTER TABLE example ADD COLUMN IF NOT EXISTS is_active boolean NOT NULL DEFAULT true; - --- Rename column (safe: metadata-only change) --- ALTER TABLE example RENAME COLUMN old_name TO new_name; - --- ------------------------------------ --- 2. Constraints --- ------------------------------------ - --- Add NOT NULL (requires all existing rows to satisfy it) --- ALTER TABLE example ALTER COLUMN name SET NOT NULL; - --- Add check constraint (NOT VALID avoids full table scan, then VALIDATE separately) --- ALTER TABLE example ADD CONSTRAINT chk_example_name CHECK (name <> '') NOT VALID; --- ALTER TABLE example VALIDATE CONSTRAINT chk_example_name; - --- Add foreign key (NOT VALID + VALIDATE pattern to avoid long locks) --- ALTER TABLE example ADD CONSTRAINT fk_example_parent --- FOREIGN KEY (parent_id) REFERENCES parent(id) NOT VALID; --- ALTER TABLE example VALIDATE CONSTRAINT fk_example_parent; - --- ------------------------------------ --- 3. Indexes (use CONCURRENTLY outside transaction) --- ------------------------------------ - --- NOTE: CREATE INDEX CONCURRENTLY cannot run inside a transaction. --- Run these statements separately after committing the transaction above. --- --- CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_example_name --- ON example (name); --- --- CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_example_created_at --- ON example USING brin (created_at); - --- ------------------------------------ --- 4. Data migration --- ------------------------------------ - --- Backfill in batches to avoid long transactions: --- UPDATE example SET description = 'default' WHERE description IS NULL; --- --- For large tables, batch with: --- DO $$ --- DECLARE --- batch_size int := 10000; --- rows_updated int; --- BEGIN --- LOOP --- UPDATE example --- SET description = 'default' --- WHERE id IN ( --- SELECT id FROM example --- WHERE description IS NULL --- LIMIT batch_size --- FOR UPDATE SKIP LOCKED --- ); --- GET DIAGNOSTICS rows_updated = ROW_COUNT; --- EXIT WHEN rows_updated = 0; --- RAISE NOTICE 'Updated % rows', rows_updated; --- COMMIT; --- END LOOP; --- END $$; - --- ------------------------------------ --- 5. Permissions --- ------------------------------------ - --- GRANT SELECT, INSERT, UPDATE ON example TO app_role; --- GRANT USAGE ON SEQUENCE example_id_seq TO app_role; - -COMMIT; - - --- ============================================================ --- DOWN MIGRATION (rollback) --- ============================================================ --- Run this section to undo the UP migration. --- Test this independently before deploying the UP migration. - --- BEGIN; --- --- -- Reverse data migration --- -- UPDATE example SET description = NULL; --- --- -- Drop constraints --- -- ALTER TABLE example DROP CONSTRAINT IF EXISTS chk_example_name; --- -- ALTER TABLE example DROP CONSTRAINT IF EXISTS fk_example_parent; --- --- -- Drop columns --- -- ALTER TABLE example DROP COLUMN IF EXISTS description; --- -- ALTER TABLE example DROP COLUMN IF EXISTS is_active; --- --- -- Drop tables --- -- DROP TABLE IF EXISTS example; --- --- COMMIT; --- --- -- Drop indexes (outside transaction) --- -- DROP INDEX CONCURRENTLY IF EXISTS idx_example_name; --- -- DROP INDEX CONCURRENTLY IF EXISTS idx_example_created_at; diff --git a/.claude/skills/databases/references/migrations.md b/.claude/skills/databases/references/migrations.md new file mode 100644 index 0000000..d407c10 --- /dev/null +++ b/.claude/skills/databases/references/migrations.md @@ -0,0 +1,312 @@ +# Databases — Migration Patterns + + +# Database Migrations + +## When to Use + +- Adding or modifying database tables/columns +- Creating indexes or constraints +- Running migrations in development, staging, or production +- Resolving migration conflicts in a team +- Rolling back a failed migration + +## When NOT to Use + +- Query optimization without schema changes — use `postgresql` skill +- Initial database design from scratch — use `postgresql` or `mongodb` skill +- ORM configuration without migrations — use framework-specific skill + +--- + +## Quick Reference + +| I need... | Go to | +|-----------|-------| +| Alembic (FastAPI/SQLAlchemy) | SS Alembic below | +| Prisma (NestJS/Express) | SS Prisma below | +| Django migrations | SS Django below | +| Safe production patterns | SS Production Safety below | +| Rollback strategies | SS Rollbacks below | + +--- + +## Alembic (Python / SQLAlchemy) + +### Setup + +```bash +pip install alembic +alembic init migrations +``` + +```python +# migrations/env.py — configure target metadata +from src.models import Base +target_metadata = Base.metadata +``` + +### Create a migration + +```bash +# Auto-generate from model changes +alembic revision --autogenerate -m "add orders table" + +# Manual migration (for data migrations or complex changes) +alembic revision -m "backfill order status" +``` + +### Migration file + +```python +# migrations/versions/003_add_orders_table.py +"""add orders table""" + +from alembic import op +import sqlalchemy as sa + +revision = '003' +down_revision = '002' + +def upgrade() -> None: + op.create_table( + 'orders', + sa.Column('id', sa.UUID(), primary_key=True, server_default=sa.text('gen_random_uuid()')), + sa.Column('user_id', sa.UUID(), sa.ForeignKey('users.id', ondelete='CASCADE'), nullable=False), + sa.Column('total', sa.Numeric(10, 2), nullable=False), + sa.Column('status', sa.String(20), nullable=False, server_default='pending'), + sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.func.now()), + ) + op.create_index('ix_orders_user_id', 'orders', ['user_id']) + op.create_index('ix_orders_created_at', 'orders', ['created_at']) + +def downgrade() -> None: + op.drop_table('orders') +``` + +### Run migrations + +```bash +# Apply all pending +alembic upgrade head + +# Apply one step +alembic upgrade +1 + +# Check current state +alembic current + +# Check for pending migrations +alembic check + +# View migration history +alembic history --verbose +``` + +--- + +## Prisma (TypeScript / NestJS / Express) + +### Create a migration + +```bash +# Generate migration from schema changes +npx prisma migrate dev --name add_orders_table + +# Apply in production (no interactive prompts) +npx prisma migrate deploy + +# Check status +npx prisma migrate status +``` + +### Schema change + +```prisma +// prisma/schema.prisma +model Order { + id String @id @default(uuid()) + userId String + user User @relation(fields: [userId], references: [id], onDelete: Cascade) + total Decimal @db.Decimal(10, 2) + status String @default("pending") + createdAt DateTime @default(now()) + + @@index([userId]) + @@index([createdAt]) +} +``` + +### Generated migration SQL + +```sql +-- prisma/migrations/20260417_add_orders_table/migration.sql +CREATE TABLE "Order" ( + "id" TEXT NOT NULL DEFAULT gen_random_uuid(), + "userId" TEXT NOT NULL, + "total" DECIMAL(10,2) NOT NULL, + "status" TEXT NOT NULL DEFAULT 'pending', + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + CONSTRAINT "Order_pkey" PRIMARY KEY ("id") +); + +CREATE INDEX "Order_userId_idx" ON "Order"("userId"); +CREATE INDEX "Order_createdAt_idx" ON "Order"("createdAt"); + +ALTER TABLE "Order" ADD CONSTRAINT "Order_userId_fkey" + FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE; +``` + +--- + +## Django + +### Create and apply + +```bash +# Auto-generate from model changes +python manage.py makemigrations app_name + +# Apply +python manage.py migrate + +# Check for pending +python manage.py showmigrations + +# SQL preview (don't execute) +python manage.py sqlmigrate app_name 0003 +``` + +### Data migration + +```python +# app/migrations/0004_backfill_order_status.py +from django.db import migrations + +def backfill_status(apps, schema_editor): + Order = apps.get_model('orders', 'Order') + Order.objects.filter(status='').update(status='pending') + +class Migration(migrations.Migration): + dependencies = [('orders', '0003_add_orders')] + operations = [migrations.RunPython(backfill_status, migrations.RunPython.noop)] +``` + +--- + +## Production Safety + +### Golden rules + +1. **Never drop columns in the same deploy as removing code references.** Remove code first, deploy, then drop column in next migration. +2. **Add columns as nullable or with defaults.** `NOT NULL` without a default locks the table during backfill on large tables. +3. **Create indexes concurrently** (PostgreSQL): + ```sql + CREATE INDEX CONCURRENTLY ix_orders_status ON orders(status); + ``` +4. **Test migrations against a production-size dataset** before deploying. +5. **Always have a rollback plan** — either a `downgrade()` function or a manual SQL script. + +### Safe column addition pattern + +```python +# Step 1: Add nullable column (fast, no lock) +op.add_column('users', sa.Column('phone', sa.String(20), nullable=True)) + +# Step 2: Backfill in batches (separate migration or script) +# Don't do UPDATE users SET phone = '...' on millions of rows at once + +# Step 3: Add NOT NULL constraint (after backfill confirms all rows filled) +op.alter_column('users', 'phone', nullable=False) +``` + +### Safe column rename pattern + +``` +Deploy 1: Add new column, write to both old and new +Deploy 2: Backfill new column from old, read from new +Deploy 3: Stop writing to old column +Deploy 4: Drop old column +``` + +--- + +## Rollbacks + +### Alembic + +```bash +# Rollback one step +alembic downgrade -1 + +# Rollback to specific revision +alembic downgrade 002 + +# Rollback to base (dangerous — drops everything) +alembic downgrade base +``` + +### Prisma + +Prisma doesn't have built-in rollback. Options: +- Apply a new migration that reverses the change +- Manually run SQL: `npx prisma db execute --file rollback.sql` +- Restore from database backup + +### Django + +```bash +# Rollback to specific migration +python manage.py migrate app_name 0002 +``` + +--- + +## Team Workflow + +### Resolving migration conflicts + +When two developers create migrations from the same parent: + +**Alembic:** +```bash +# Developer A and B both branched from revision 002 +# Alembic detects multiple heads +alembic heads # shows 003a and 003b +alembic merge -m "merge migrations" 003a 003b +alembic upgrade head +``` + +**Prisma:** +```bash +# Reset and re-apply (dev only) +npx prisma migrate reset +# Or resolve manually by editing the migration SQL +``` + +**Django:** +```bash +# Django auto-detects and asks to merge +python manage.py makemigrations --merge +``` + +--- + +## Common Pitfalls + +1. **Running `migrate reset` in production.** This drops all data. Only use in development. +2. **Editing already-applied migrations.** Never modify a migration that's been deployed. Create a new migration instead. +3. **Forgetting indexes.** Add indexes for foreign keys and frequently-queried columns in the same migration. +4. **Large table locks.** `ALTER TABLE` with `NOT NULL` or `ADD COLUMN DEFAULT` can lock large tables. Use batched backfills. +5. **Not testing downgrade.** Always test your rollback path before deploying. +6. **Circular foreign keys.** Use `sa.ForeignKey` with `use_alter=True` in Alembic to handle circular deps. + +--- + +## Related Skills + +- `postgresql` — Database design, query optimization, indexing strategies +- `fastapi` — SQLAlchemy async patterns with FastAPI +- `nestjs` — Prisma integration with NestJS +- `django` — Django ORM models and migrations +- `docker` — Running migration containers in CI/CD diff --git a/.claude/skills/databases/mongodb/SKILL.md b/.claude/skills/databases/references/mongodb.md similarity index 96% rename from .claude/skills/databases/mongodb/SKILL.md rename to .claude/skills/databases/references/mongodb.md index bc736ad..6b777b2 100644 --- a/.claude/skills/databases/mongodb/SKILL.md +++ b/.claude/skills/databases/references/mongodb.md @@ -1,8 +1,5 @@ ---- -name: mongodb -description: > - Use this skill whenever working with MongoDB, document databases, or NoSQL data modeling. Trigger on keywords like MongoDB, Mongo, document database, aggregation pipeline, collection, embedded documents, or BSON. Also applies when designing document schemas, building aggregation queries, handling unstructured or semi-structured data, or migrating from relational to document-based storage. ---- +# Databases — MongoDB Patterns + # MongoDB @@ -574,6 +571,6 @@ db.orders.updateMany( ## Related Skills -- `databases/postgresql` - Relational database patterns for structured data with complex relationships -- `patterns/caching` - Caching strategies to reduce database load -- `patterns/logging` - Logging patterns for query debugging and monitoring +- `postgresql` - Relational database patterns for structured data with complex relationships +- `caching` - Caching strategies to reduce database load +- `logging` - Logging patterns for query debugging and monitoring diff --git a/.claude/skills/databases/postgresql/SKILL.md b/.claude/skills/databases/references/postgresql.md similarity index 96% rename from .claude/skills/databases/postgresql/SKILL.md rename to .claude/skills/databases/references/postgresql.md index ae8b6c6..0b2b4bc 100644 --- a/.claude/skills/databases/postgresql/SKILL.md +++ b/.claude/skills/databases/references/postgresql.md @@ -1,8 +1,5 @@ ---- -name: postgresql -description: > - Use this skill whenever working with PostgreSQL databases, writing SQL queries, designing schemas, or optimizing database performance. Trigger on keywords like PostgreSQL, Postgres, SQL query, schema design, indexing, migrations, EXPLAIN ANALYZE, connection pooling, or any relational database operation. Also applies when debugging slow queries, setting up database tables, or working with ORMs that target PostgreSQL. ---- +# Databases — PostgreSQL Patterns + # PostgreSQL @@ -607,6 +604,6 @@ SELECT * FROM orders WHERE id > 100000 ORDER BY id LIMIT 20; ## Related Skills -- `databases/mongodb` - Document-based database patterns for non-relational data -- `patterns/caching` - Caching strategies to reduce database load -- `patterns/logging` - Logging patterns for query debugging and monitoring +- `mongodb` - Document-based database patterns for non-relational data +- `caching` - Caching strategies to reduce database load +- `logging` - Logging patterns for query debugging and monitoring diff --git a/.claude/skills/databases/references/redis.md b/.claude/skills/databases/references/redis.md new file mode 100644 index 0000000..f90f485 --- /dev/null +++ b/.claude/skills/databases/references/redis.md @@ -0,0 +1,279 @@ +# Databases — Redis Patterns + + +# Redis + +## When to Use + +- Caching database queries or API responses +- Session storage for web applications +- Rate limiting (distributed across instances) +- Job/task queues (BullMQ, Celery) +- Pub/sub messaging between services +- Distributed locks + +## When NOT to Use + +- **Primary data storage** — Redis is a cache/broker, not a database of record +- **Complex queries** — use PostgreSQL for relational queries +- **Large blobs** — use S3/R2 for file storage +- **In-memory caching only** — use `functools.lru_cache` or `Map` for single-process caches + +--- + +## Python (redis-py / FastAPI) + +### Connection + +```python +# src/core/redis.py +import redis.asyncio as redis + +pool = redis.ConnectionPool.from_url( + "redis://localhost:6379/0", + max_connections=20, + decode_responses=True, +) + +async def get_redis() -> redis.Redis: + return redis.Redis(connection_pool=pool) +``` + +### Cache-aside pattern + +```python +import json +from datetime import timedelta + +async def get_user_cached(user_id: str, db: AsyncSession) -> User: + r = await get_redis() + cache_key = f"user:{user_id}" + + # Check cache + cached = await r.get(cache_key) + if cached: + return User(**json.loads(cached)) + + # Cache miss — fetch from DB + user = await db.get(User, user_id) + if not user: + raise HTTPException(status_code=404, detail="User not found") + + # Store in cache with TTL + await r.setex(cache_key, timedelta(minutes=15), json.dumps(user.to_dict())) + return user +``` + +### Cache invalidation + +```python +async def update_user(user_id: str, data: UpdateUserRequest, db: AsyncSession) -> User: + user = await db.get(User, user_id) + for key, value in data.dict(exclude_unset=True).items(): + setattr(user, key, value) + await db.commit() + + # Invalidate cache + r = await get_redis() + await r.delete(f"user:{user_id}") + + return user +``` + +### Rate limiting + +```python +from fastapi import Request, HTTPException + +async def rate_limit(request: Request, limit: int = 100, window: int = 900): + r = await get_redis() + key = f"rate:{request.client.host}" + current = await r.incr(key) + if current == 1: + await r.expire(key, window) + if current > limit: + raise HTTPException(status_code=429, detail="Rate limit exceeded") +``` + +### Session storage + +```python +import secrets + +async def create_session(user_id: str) -> str: + r = await get_redis() + session_id = secrets.token_urlsafe(32) + await r.setex(f"session:{session_id}", timedelta(hours=24), user_id) + return session_id + +async def get_session(session_id: str) -> str | None: + r = await get_redis() + return await r.get(f"session:{session_id}") + +async def delete_session(session_id: str): + r = await get_redis() + await r.delete(f"session:{session_id}") +``` + +--- + +## TypeScript (ioredis / NestJS / Express) + +### Connection + +```typescript +// src/core/redis.ts +import Redis from 'ioredis'; + +export const redis = new Redis(process.env.REDIS_URL ?? 'redis://localhost:6379', { + maxRetriesPerRequest: 3, + lazyConnect: true, +}); +``` + +### NestJS module + +```typescript +// src/cache/cache.module.ts +import { Global, Module } from '@nestjs/common'; +import { CacheService } from './cache.service'; + +@Global() +@Module({ + providers: [CacheService], + exports: [CacheService], +}) +export class CacheModule {} +``` + +```typescript +// src/cache/cache.service.ts +import { Injectable, OnModuleDestroy } from '@nestjs/common'; +import Redis from 'ioredis'; + +@Injectable() +export class CacheService implements OnModuleDestroy { + private readonly redis = new Redis(process.env.REDIS_URL!); + + async get(key: string): Promise { + const data = await this.redis.get(key); + return data ? JSON.parse(data) : null; + } + + async set(key: string, value: unknown, ttlSeconds: number): Promise { + await this.redis.setex(key, ttlSeconds, JSON.stringify(value)); + } + + async del(key: string): Promise { + await this.redis.del(key); + } + + async onModuleDestroy() { + await this.redis.quit(); + } +} +``` + +### Cache-aside in service + +```typescript +@Injectable() +export class UsersService { + constructor( + private readonly prisma: PrismaService, + private readonly cache: CacheService, + ) {} + + async findOne(id: string): Promise { + // Check cache + const cached = await this.cache.get(`user:${id}`); + if (cached) return cached; + + // Cache miss + const user = await this.prisma.user.findUnique({ where: { id } }); + if (!user) throw new NotFoundException(`User ${id} not found`); + + // Store with 15min TTL + await this.cache.set(`user:${id}`, user, 900); + return user; + } + + async update(id: string, dto: UpdateUserDto): Promise { + const user = await this.prisma.user.update({ where: { id }, data: dto }); + await this.cache.del(`user:${id}`); // Invalidate + return user; + } +} +``` + +--- + +## Pub/Sub + +### Python + +```python +# Publisher +async def publish_event(channel: str, event: dict): + r = await get_redis() + await r.publish(channel, json.dumps(event)) + +# Subscriber +async def subscribe_events(channel: str): + r = await get_redis() + pubsub = r.pubsub() + await pubsub.subscribe(channel) + async for message in pubsub.listen(): + if message['type'] == 'message': + yield json.loads(message['data']) +``` + +### TypeScript + +```typescript +// Publisher +const pub = new Redis(process.env.REDIS_URL!); +await pub.publish('orders', JSON.stringify({ type: 'created', orderId: '123' })); + +// Subscriber (separate connection required) +const sub = new Redis(process.env.REDIS_URL!); +sub.subscribe('orders'); +sub.on('message', (channel, message) => { + const event = JSON.parse(message); + console.log(`[${channel}]`, event); +}); +``` + +--- + +## Key Naming Conventions + +``` +entity:id → user:abc123 +entity:id:field → user:abc123:orders +rate:ip → rate:192.168.1.1 +session:token → session:abc123def +lock:resource → lock:order-processing +queue:name → queue:email-notifications +``` + +--- + +## Common Pitfalls + +1. **Not setting TTLs.** Every cache key should have an expiration. Unbounded caches exhaust memory. +2. **Cache stampede.** When a popular key expires, many requests hit the DB simultaneously. Use distributed locks or stale-while-revalidate. +3. **Using the same connection for pub/sub.** Subscribers can't run other commands. Use a dedicated connection. +4. **Storing large objects.** Redis is fast for small values. Keep values under 1MB; for larger data, store a pointer to S3. +5. **Not handling connection failures.** Redis connections drop. Use retry logic and connection pools. +6. **Forgetting to invalidate.** When data changes, delete the cache key. Stale cache is worse than no cache. + +--- + +## Related Skills + +- `caching` — HTTP caching, CDN, memoization (framework-agnostic patterns) +- `background-jobs` — BullMQ/Celery use Redis as broker +- `fastapi` — Redis integration with FastAPI dependency injection +- `nestjs` — Redis caching module in NestJS +- `docker` — Running Redis in Docker Compose for development diff --git a/.claude/skills/methodology/defense-in-depth/SKILL.md b/.claude/skills/defense-in-depth/SKILL.md similarity index 88% rename from .claude/skills/methodology/defense-in-depth/SKILL.md rename to .claude/skills/defense-in-depth/SKILL.md index a458f0b..6c9fb18 100644 --- a/.claude/skills/methodology/defense-in-depth/SKILL.md +++ b/.claude/skills/defense-in-depth/SKILL.md @@ -1,7 +1,7 @@ --- name: defense-in-depth description: > - Trigger this skill after fixing any data-related bug, when building validation for critical data paths, or when a single validation point has already failed in production. Use whenever you hear "it slipped through," "the check was bypassed," or "it worked in tests but not production." Apply aggressively to any scenario involving data integrity, input validation across layers, or preventing bug recurrence through structural guarantees rather than single-point fixes. + Use when fixing any data-related bug, when building validation for critical data paths, or when a single validation point has already failed in production. Also activate whenever you hear "it slipped through," "the check was bypassed," or "it worked in tests but not production." Apply aggressively to any scenario involving data integrity, input validation across layers, or preventing bug recurrence through structural guarantees rather than single-point fixes. --- # Defense-in-Depth @@ -294,6 +294,6 @@ After fixing any bug: ## Related Skills -- `methodology/root-cause-tracing` - Use before defense-in-depth to find the actual source of the bug before adding multi-layer validation -- `methodology/systematic-debugging` - General debugging methodology that pairs with defense-in-depth for comprehensive bug resolution -- `security/owasp` - Security-specific validation patterns that complement defense-in-depth for security-sensitive code paths +- `root-cause-tracing` - Use before defense-in-depth to find the actual source of the bug before adding multi-layer validation +- `systematic-debugging` - General debugging methodology that pairs with defense-in-depth for comprehensive bug resolution +- `owasp` - Security-specific validation patterns that complement defense-in-depth for security-sensitive code paths diff --git a/.claude/skills/methodology/defense-in-depth/references/validation-layers.md b/.claude/skills/defense-in-depth/references/validation-layers.md similarity index 100% rename from .claude/skills/methodology/defense-in-depth/references/validation-layers.md rename to .claude/skills/defense-in-depth/references/validation-layers.md diff --git a/.claude/skills/devops/SKILL.md b/.claude/skills/devops/SKILL.md new file mode 100644 index 0000000..964b422 --- /dev/null +++ b/.claude/skills/devops/SKILL.md @@ -0,0 +1,64 @@ +--- +name: devops +description: > + Use when containerizing applications, configuring CI/CD pipelines, or deploying to edge — including Docker, Dockerfile, docker-compose, multi-stage builds, GitHub Actions, workflow YAML, matrix builds, workflow_dispatch, Cloudflare Workers, Pages, R2, D1, KV, wrangler, or container registries. +--- + +# DevOps + +## When to Use + +- Containerizing applications with Docker or Docker Compose +- Setting up CI/CD pipelines with GitHub Actions +- Deploying to Cloudflare Workers, Pages, R2, D1, or KV +- Optimizing container images, build caching, or deployment workflows +- Configuring wrangler.toml, Durable Objects, or Cloudflare Queues + +## When NOT to Use + +- Application code without infrastructure concerns — use framework-specific skills +- Database schema changes — use `databases` +- Security auditing — use `owasp` + +--- + +## Quick Reference + +| Topic | Reference | Key features | +|-------|-----------|-------------| +| Docker | `references/docker.md` | Dockerfiles, multi-stage builds, Compose, .dockerignore, healthchecks | +| GitHub Actions | `references/github-actions.md` | Workflow YAML, matrix builds, caching, secrets, reusable workflows | +| Cloudflare Workers | `references/cloudflare-workers.md` | Workers, Pages, R2, D1, KV, Durable Objects, wrangler | + +--- + +## Best Practices + +1. **Use multi-stage builds** to keep production images small (Docker). +2. **Pin image tags and action versions** — use digests or major version tags, never `latest`. +3. **Order instructions for cache efficiency** — copy dependency manifests before application code (Docker). +4. **Run as non-root** in containers (Docker). +5. **Use caching aggressively** in CI — cache package manager stores and Docker layers (GitHub Actions). +6. **Set minimal permissions** — add a top-level `permissions` block (GitHub Actions). +7. **Extract reusable workflows and composite actions** for shared CI logic (GitHub Actions). +8. **Keep secrets out of logs** — never `echo` a secret (GitHub Actions). + +## Common Pitfalls + +1. **Bloated images** — using full base images instead of slim/alpine variants (Docker). +2. **Cache invalidation by COPY order** — placing `COPY . .` before `RUN pip install` (Docker). +3. **Secrets baked into layers** (Docker). +4. **Unpinned action versions** (GitHub Actions). +5. **Overly broad triggers** — triggering on every push to every branch (GitHub Actions). +6. **Secret exposure in pull requests from forks** (GitHub Actions). +7. **Using Node.js APIs without `nodejs_compat`** (Cloudflare Workers). +8. **Blocking the event loop** — Workers have strict CPU time limits (Cloudflare Workers). +9. **Using KV for frequently updated data** — eventually consistent with ~60s propagation (Cloudflare Workers). + +--- + +## Related Skills + +- `backend-frameworks` — Application code that gets containerized +- `databases` — Database services in Docker Compose +- `owasp` — Security hardening for containers and CI diff --git a/.claude/skills/devops/docker/references/dockerfile-best-practices.md b/.claude/skills/devops/docker/references/dockerfile-best-practices.md deleted file mode 100644 index 96419b7..0000000 --- a/.claude/skills/devops/docker/references/dockerfile-best-practices.md +++ /dev/null @@ -1,196 +0,0 @@ -# Dockerfile Best Practices Reference - -Quick reference for writing efficient, secure, and maintainable Dockerfiles. - -## Layer Ordering for Cache Optimization - -Order instructions from least-frequently-changed to most-frequently-changed: - -```dockerfile -# 1. Base image (changes: rarely) -FROM node:22-slim - -# 2. System dependencies (changes: rarely) -RUN apt-get update && apt-get install -y --no-install-recommends \ - curl \ - && rm -rf /var/lib/apt/lists/* - -# 3. App dependency manifest (changes: sometimes) -COPY package.json pnpm-lock.yaml ./ - -# 4. Install dependencies (changes: sometimes, cached if manifests unchanged) -RUN pnpm install --frozen-lockfile - -# 5. Copy source code (changes: frequently) -COPY . . - -# 6. Build step (changes: frequently) -RUN pnpm build - -# 7. Runtime config (changes: rarely, but placed last for clarity) -EXPOSE 3000 -CMD ["node", "dist/server.js"] -``` - -**Key rule**: If a layer changes, all subsequent layers are rebuilt. Separate dependency installation from source code copying. - -## Multi-Stage Builds - -Reduce final image size by separating build and runtime stages. - -``` -+-------------------+ +-------------------+ -| Build Stage | | Runtime Stage | -| | | | -| - Full toolchain | ---> | - Minimal base | -| - Dev deps | | - Only artifacts | -| - Source code | | - No build tools | -| - Build output | | - No source code | -+-------------------+ +-------------------+ - ~800 MB ~80 MB -``` - -**Benefits**: Smaller images, faster deploys, reduced attack surface, no build tools in production. - -## Base Image Selection - -| Image | Size | Use Case | Security | Package Manager | -|-------|------|----------|----------|-----------------| -| **alpine** | ~5 MB | Small images, CLI tools | Good (small surface) | apk | -| **slim** (Debian) | ~80 MB | Most apps (Python, Node) | Good | apt | -| **distroless** | ~20 MB | Production, no shell needed | Excellent (no shell) | None | -| **scratch** | 0 MB | Static Go/Rust binaries | Excellent (nothing) | None | -| **full** (Debian) | ~300 MB | Build stages, debugging | Fair (large surface) | apt | - -### Recommendations by Language - -| Language | Build Stage | Runtime Stage | -|----------|-------------|---------------| -| **Python** | `python:3.12-slim` | `python:3.12-slim` or `distroless/python3` | -| **Node.js** | `node:22-slim` | `node:22-slim` or `distroless/nodejs22` | -| **Go** | `golang:1.23` | `scratch` or `distroless/static` | -| **Rust** | `rust:1.83` | `scratch` or `distroless/cc` | -| **Java** | `eclipse-temurin:21-jdk` | `eclipse-temurin:21-jre-alpine` | - -## Instruction Best Practices - -### RUN: Combine and Clean Up - -```dockerfile -# BAD: Multiple layers, leftover cache -RUN apt-get update -RUN apt-get install -y curl -RUN apt-get install -y git - -# GOOD: Single layer, cache cleaned -RUN apt-get update && apt-get install -y --no-install-recommends \ - curl \ - git \ - && rm -rf /var/lib/apt/lists/* -``` - -### COPY: Be Specific - -```dockerfile -# BAD: Copies everything, including .git, node_modules, etc. -COPY . . - -# GOOD: Copy only what's needed (use .dockerignore too) -COPY package.json pnpm-lock.yaml ./ -RUN pnpm install --frozen-lockfile -COPY src/ ./src/ -COPY tsconfig.json ./ -``` - -### .dockerignore Essentials - -``` -.git -node_modules -__pycache__ -.env -*.log -dist -.venv -.pytest_cache -coverage -.DS_Store -``` - -### USER: Don't Run as Root - -```dockerfile -# Create non-root user -RUN groupadd -r appuser && useradd -r -g appuser -s /bin/false appuser -USER appuser -``` - -### HEALTHCHECK - -```dockerfile -HEALTHCHECK --interval=30s --timeout=3s --retries=3 \ - CMD curl -f http://localhost:3000/health || exit 1 -``` - -### ARG vs ENV - -| Directive | Available at | Persists in image | Use for | -|-----------|-------------|-------------------|---------| -| `ARG` | Build time only | No | Build-time toggles, versions | -| `ENV` | Build + runtime | Yes | App configuration | - -```dockerfile -ARG PYTHON_VERSION=3.12 -FROM python:${PYTHON_VERSION}-slim - -ENV APP_ENV=production -ENV PORT=8000 -``` - -## Security Checklist - -| Practice | Command/Example | -|----------|----------------| -| Pin base image digests | `FROM node:22-slim@sha256:abc123...` | -| Run as non-root | `USER appuser` | -| No secrets in layers | Use `--mount=type=secret` or build args | -| Scan for vulnerabilities | `docker scout cves`, `trivy image` | -| Read-only filesystem | `docker run --read-only` | -| Drop capabilities | `docker run --cap-drop ALL` | -| Use `.dockerignore` | Exclude `.env`, `.git`, credentials | -| Minimal base image | Use slim/distroless/scratch | - -### Secrets at Build Time (BuildKit) - -```dockerfile -# Mount a secret file without baking it into a layer -RUN --mount=type=secret,id=npm_token \ - NPM_TOKEN=$(cat /run/secrets/npm_token) \ - npm install - -# Build command: -# docker build --secret id=npm_token,src=.npmrc . -``` - -## Image Size Reduction Checklist - -1. Use multi-stage builds -2. Choose slim/alpine/distroless base -3. Combine RUN commands -4. Remove package manager caches (`rm -rf /var/lib/apt/lists/*`) -5. Use `.dockerignore` -6. Don't install dev dependencies in runtime stage -7. Remove unnecessary files after build -8. Use `--no-install-recommends` with apt - -## Common Pitfalls - -| Pitfall | Impact | Fix | -|---------|--------|-----| -| `COPY . .` before `npm install` | No dependency caching | Copy lockfile first, install, then copy source | -| Using `latest` tag | Non-reproducible builds | Pin specific version tags or digests | -| Secrets in `ENV` or `COPY` | Leaked in image layers | Use BuildKit secrets mount | -| Running as root | Security vulnerability | Add `USER` directive | -| No `.dockerignore` | Bloated context, slow builds | Add and maintain `.dockerignore` | -| Installing build tools in final stage | Bloated image | Use multi-stage; build in first stage | -| Not using `--frozen-lockfile` | Non-deterministic installs | Always use lockfile flags | diff --git a/.claude/skills/devops/docker/templates/Dockerfile.node b/.claude/skills/devops/docker/templates/Dockerfile.node deleted file mode 100644 index b505b47..0000000 --- a/.claude/skills/devops/docker/templates/Dockerfile.node +++ /dev/null @@ -1,93 +0,0 @@ -# ============================================================================= -# Multi-Stage Node.js Dockerfile -# Usage: -# docker build -t myapp . -# docker run -p 3000:3000 myapp -# ============================================================================= - -# --------------------------------------------------------------------------- -# Stage 1: Install dependencies -# --------------------------------------------------------------------------- -FROM node:22-slim AS deps - -# Enable corepack for pnpm support. -RUN corepack enable - -WORKDIR /app - -# Copy only package manifests first for dependency layer caching. -# Dependencies are only reinstalled when these files change. -COPY package.json pnpm-lock.yaml ./ - -# Install production and dev dependencies (dev deps needed for build step). -RUN pnpm install --frozen-lockfile - -# --------------------------------------------------------------------------- -# Stage 2: Build the application -# --------------------------------------------------------------------------- -FROM node:22-slim AS builder - -RUN corepack enable - -WORKDIR /app - -# Copy dependencies from the deps stage. -COPY --from=deps /app/node_modules ./node_modules -COPY --from=deps /app/package.json /app/pnpm-lock.yaml ./ - -# Copy source code and config files needed for the build. -COPY tsconfig.json ./ -COPY src/ ./src/ -# COPY public/ ./public/ # Uncomment for Next.js or static assets - -# Build the application. -RUN pnpm build - -# Remove dev dependencies after build to reduce size. -RUN pnpm prune --prod - -# --------------------------------------------------------------------------- -# Stage 3: Production runtime -# --------------------------------------------------------------------------- -FROM node:22-slim AS runtime - -# Run as non-root for security. -RUN groupadd -r appuser && useradd -r -g appuser -d /app -s /bin/false appuser - -WORKDIR /app - -# Set production environment. -ENV NODE_ENV=production \ - PORT=3000 - -# Copy only production artifacts from the builder stage. -COPY --from=builder /app/node_modules ./node_modules -COPY --from=builder /app/dist ./dist -COPY --from=builder /app/package.json ./ - -# For Next.js standalone output, use instead: -# COPY --from=builder /app/.next/standalone ./ -# COPY --from=builder /app/.next/static ./.next/static -# COPY --from=builder /app/public ./public - -# Switch to non-root user. -USER appuser - -# Expose the application port. -EXPOSE 3000 - -# Health check -- adjust the endpoint to match your app. -HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \ - CMD node -e "fetch('http://localhost:3000/health').then(r => { if (!r.ok) process.exit(1) })" || exit 1 - -# Run the application. -CMD ["node", "dist/server.js"] - -# For Next.js standalone: -# CMD ["node", "server.js"] - -# For NestJS: -# CMD ["node", "dist/main.js"] - -# For Express with ts-node (dev only, not recommended for production): -# CMD ["npx", "ts-node", "src/server.ts"] diff --git a/.claude/skills/devops/docker/templates/Dockerfile.python b/.claude/skills/devops/docker/templates/Dockerfile.python deleted file mode 100644 index 036cfa0..0000000 --- a/.claude/skills/devops/docker/templates/Dockerfile.python +++ /dev/null @@ -1,78 +0,0 @@ -# ============================================================================= -# Multi-Stage Python Dockerfile -# Usage: -# docker build -t myapp . -# docker run -p 8000:8000 myapp -# ============================================================================= - -# --------------------------------------------------------------------------- -# Stage 1: Build dependencies -# --------------------------------------------------------------------------- -# Use slim for building - it has gcc and headers available via apt. -FROM python:3.12-slim AS builder - -# Prevent Python from writing .pyc files and enable unbuffered output. -ENV PYTHONDONTWRITEBYTECODE=1 \ - PYTHONUNBUFFERED=1 - -# Install build-time system dependencies (if any compiled packages need them). -RUN apt-get update && apt-get install -y --no-install-recommends \ - build-essential \ - && rm -rf /var/lib/apt/lists/* - -WORKDIR /app - -# Install Python dependencies into a virtual environment. -# Copying requirements first enables Docker layer caching -- -# dependencies are only reinstalled when requirements.txt changes. -COPY requirements.txt . -RUN python -m venv /app/.venv \ - && /app/.venv/bin/pip install --no-cache-dir --upgrade pip \ - && /app/.venv/bin/pip install --no-cache-dir -r requirements.txt - -# --------------------------------------------------------------------------- -# Stage 2: Runtime -# --------------------------------------------------------------------------- -FROM python:3.12-slim AS runtime - -# Prevent .pyc files and enable unbuffered output for logging. -ENV PYTHONDONTWRITEBYTECODE=1 \ - PYTHONUNBUFFERED=1 \ - PATH="/app/.venv/bin:$PATH" - -# Install only runtime system dependencies (no build tools). -# Add packages here if your app needs them at runtime (e.g., libpq for psycopg). -# RUN apt-get update && apt-get install -y --no-install-recommends \ -# libpq5 \ -# && rm -rf /var/lib/apt/lists/* - -# Create a non-root user for security. -RUN groupadd -r appuser && useradd -r -g appuser -d /app -s /bin/false appuser - -WORKDIR /app - -# Copy the virtual environment from the builder stage. -COPY --from=builder /app/.venv /app/.venv - -# Copy application source code. -COPY src/ ./src/ - -# Switch to non-root user. -USER appuser - -# Expose the application port. -EXPOSE 8000 - -# Health check -- adjust the endpoint to match your app. -HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \ - CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1 - -# Run the application. -# For FastAPI/Uvicorn: -CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8000"] - -# For Django/Gunicorn: -# CMD ["gunicorn", "src.wsgi:application", "--bind", "0.0.0.0:8000", "--workers", "4"] - -# For a plain script: -# CMD ["python", "-m", "src.main"] diff --git a/.claude/skills/devops/docker/templates/docker-compose.dev.yaml b/.claude/skills/devops/docker/templates/docker-compose.dev.yaml deleted file mode 100644 index 4aa7159..0000000 --- a/.claude/skills/devops/docker/templates/docker-compose.dev.yaml +++ /dev/null @@ -1,100 +0,0 @@ -# ============================================================================= -# Development Docker Compose -# Usage: -# docker compose -f docker-compose.dev.yaml up -# docker compose -f docker-compose.dev.yaml down -v # remove volumes too -# ============================================================================= - -services: - # --------------------------------------------------------------------------- - # Application - # --------------------------------------------------------------------------- - app: - build: - context: . - dockerfile: Dockerfile - target: builder # Use the build stage for dev (includes dev deps) - ports: - - "${APP_PORT:-3000}:3000" - environment: - NODE_ENV: development - DATABASE_URL: postgresql://postgres:postgres@postgres:5432/app_dev - REDIS_URL: redis://redis:6379/0 - volumes: - # Mount source code for hot-reload. Exclude node_modules. - - .:/app - - /app/node_modules - depends_on: - postgres: - condition: service_healthy - redis: - condition: service_healthy - # Override CMD for development (hot-reload). - command: ["pnpm", "dev"] - # For Python: - # command: ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "3000", "--reload"] - - # --------------------------------------------------------------------------- - # PostgreSQL - # --------------------------------------------------------------------------- - postgres: - image: postgres:17-alpine - ports: - - "${POSTGRES_PORT:-5432}:5432" - environment: - POSTGRES_USER: postgres - POSTGRES_PASSWORD: postgres - POSTGRES_DB: app_dev - volumes: - # Persist data across restarts. - - postgres_data:/var/lib/postgresql/data - # Run init scripts on first start. - # - ./scripts/init-db.sql:/docker-entrypoint-initdb.d/init.sql - healthcheck: - test: ["CMD-SHELL", "pg_isready -U postgres"] - interval: 5s - timeout: 3s - retries: 5 - - # --------------------------------------------------------------------------- - # Redis - # --------------------------------------------------------------------------- - redis: - image: redis:7-alpine - ports: - - "${REDIS_PORT:-6379}:6379" - volumes: - - redis_data:/data - healthcheck: - test: ["CMD", "redis-cli", "ping"] - interval: 5s - timeout: 3s - retries: 5 - # Persist data to disk every 60 seconds if at least 1 key changed. - command: ["redis-server", "--save", "60", "1", "--loglevel", "warning"] - - # --------------------------------------------------------------------------- - # Optional: pgAdmin (database GUI) - # --------------------------------------------------------------------------- - # pgadmin: - # image: dpage/pgadmin4:latest - # ports: - # - "5050:80" - # environment: - # PGADMIN_DEFAULT_EMAIL: admin@local.dev - # PGADMIN_DEFAULT_PASSWORD: admin - # depends_on: - # - postgres - - # --------------------------------------------------------------------------- - # Optional: Mailpit (email testing) - # --------------------------------------------------------------------------- - # mailpit: - # image: axllent/mailpit:latest - # ports: - # - "8025:8025" # Web UI - # - "1025:1025" # SMTP - -volumes: - postgres_data: - redis_data: diff --git a/.claude/skills/devops/github-actions/references/gha-syntax.md b/.claude/skills/devops/github-actions/references/gha-syntax.md deleted file mode 100644 index 5d4f356..0000000 --- a/.claude/skills/devops/github-actions/references/gha-syntax.md +++ /dev/null @@ -1,250 +0,0 @@ -# GitHub Actions Syntax Quick Reference -## Workflow File Structure - -```yaml -name: CI # Workflow name (shown in GitHub UI) - -on: # Triggers - push: - branches: [main] - pull_request: - branches: [main] - -permissions: # Workflow-level permissions - contents: read - -env: # Workflow-level environment variables - NODE_ENV: test - -jobs: - build: # Job ID - runs-on: ubuntu-latest # Runner - steps: - - uses: actions/checkout@v4 # Action step - - run: echo "Hello" # Shell step -``` - -## Triggers (on:) - -### Common Events - -```yaml -on: - push: - branches: [main, "release/**"] - paths: ["src/**", "!src/**/*.test.ts"] # Path filtering - tags: ["v*"] - pull_request: - branches: [main] - types: [opened, synchronize, reopened] - workflow_dispatch: # Manual trigger - inputs: - environment: - type: choice - options: [staging, production] - schedule: - - cron: "0 6 * * 1" # Every Monday at 6am UTC - release: - types: [published] - workflow_call: # Reusable workflow - inputs: - node-version: { type: string, default: "22" } - secrets: - NPM_TOKEN: { required: true } -``` - -## Jobs - -```yaml -jobs: - test: - name: Run Tests - runs-on: ubuntu-latest - timeout-minutes: 15 - steps: - - uses: actions/checkout@v4 - - run: npm test - deploy: - needs: [lint, test] # Runs after lint AND test succeed - runs-on: ubuntu-latest - steps: [...] -``` - -### Matrix Strategy - -```yaml -jobs: - test: - strategy: - fail-fast: false # Don't cancel other jobs on failure - matrix: - os: [ubuntu-latest, macos-latest] - node: [20, 22] - exclude: - - os: macos-latest - node: 20 - include: - - os: ubuntu-latest - node: 22 - coverage: true - runs-on: ${{ matrix.os }} - steps: - - uses: actions/setup-node@v4 - with: - node-version: ${{ matrix.node }} -``` - -## Steps - -### Action Step - -```yaml -- uses: actions/checkout@v4 - with: - fetch-depth: 0 # Full history (needed for some tools) - -- uses: actions/setup-node@v4 - with: - node-version-file: ".nvmrc" - cache: "pnpm" -``` - -### Shell Step - -```yaml -- name: Run tests - run: npm test - working-directory: ./packages/api - env: - DATABASE_URL: ${{ secrets.DATABASE_URL }} - shell: bash - continue-on-error: true # Don't fail the job - timeout-minutes: 10 -``` - -### Multi-line Commands - -```yaml -- run: | - echo "Line 1" - echo "Line 2" - npm run build -``` - -## Conditionals (if:) - -```yaml -# Run only on main branch -- if: github.ref == 'refs/heads/main' - -# Run only on pull requests -- if: github.event_name == 'pull_request' - -# Run only when previous step failed -- if: failure() - -# Always run (even if previous steps failed) -- if: always() - -# Run only when a matrix variable is set -- if: matrix.coverage == true - -# Run based on changed files (requires dorny/paths-filter or similar) -- if: steps.filter.outputs.backend == 'true' - -# Run on specific actor -- if: github.actor != 'dependabot[bot]' -``` - -## Environment and Secrets - -```yaml -jobs: - deploy: - environment: - name: production - url: https://example.com - env: - APP_VERSION: ${{ github.sha }} - steps: - - run: deploy.sh - env: - API_KEY: ${{ secrets.API_KEY }} # Repository secret - DEPLOY_TOKEN: ${{ vars.DEPLOY_TOKEN }} # Repository variable -``` - -## Caching - -### Built-in Cache (setup actions) - -```yaml -- uses: actions/setup-node@v4 - with: - node-version: 22 - cache: "pnpm" # Automatic pnpm cache -``` - -### Manual Cache - -```yaml -- uses: actions/cache@v4 - with: - path: | - ~/.cache/pip - .mypy_cache - key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }} - restore-keys: | - ${{ runner.os }}-pip- -``` - -## Artifacts - -### Upload - -```yaml -- uses: actions/upload-artifact@v4 - with: - name: coverage-report - path: coverage/ - retention-days: 7 -``` - -### Download (in another job) - -```yaml -- uses: actions/download-artifact@v4 - with: - name: coverage-report - path: ./coverage -``` - -## Services (Containers) - -Define `services:` under a job with `image`, `env`, `ports`, and `options` (for health checks). Common: postgres, redis, mysql. - -## Outputs (Passing Data Between Steps/Jobs) - -```yaml -# Step output: echo "key=value" >> "$GITHUB_OUTPUT" -# Read in later step: ${{ steps..outputs.key }} -# Job output: declare under jobs..outputs, read via needs..outputs.key -``` - -## Permissions - -Common values: `contents: read`, `pull-requests: write`, `issues: write`, `packages: write`, `id-token: write` (OIDC). - -## Reusable Workflows - -Caller uses `uses: ./.github/workflows/reusable.yaml` with `with:` and `secrets: inherit`. Callee triggers on `workflow_call:` with `inputs:` and `secrets:` definitions. - -## Useful Expressions - -| Expression | Result | -|-----------|--------| -| `${{ github.sha }}` | Full commit SHA | -| `${{ github.ref_name }}` | Branch or tag name | -| `${{ github.event.pull_request.number }}` | PR number | -| `${{ runner.os }}` | `Linux`, `macOS`, `Windows` | -| `${{ hashFiles('**/lockfile') }}` | SHA256 of files | -| `${{ contains(github.event.head_commit.message, '[skip ci]') }}` | Check commit message | diff --git a/.claude/skills/devops/github-actions/templates/ci-node.yaml b/.claude/skills/devops/github-actions/templates/ci-node.yaml deleted file mode 100644 index 7a64df0..0000000 --- a/.claude/skills/devops/github-actions/templates/ci-node.yaml +++ /dev/null @@ -1,176 +0,0 @@ -# ============================================================================= -# Node.js CI Pipeline -# Runs: lint (eslint), type check (tsc), test (vitest), build -# ============================================================================= - -name: Node CI - -on: - push: - branches: [main] - paths: - - "**.ts" - - "**.tsx" - - "**.js" - - "**.jsx" - - "package.json" - - "pnpm-lock.yaml" - - "tsconfig.json" - - ".github/workflows/ci-node.yaml" - pull_request: - branches: [main] - paths: - - "**.ts" - - "**.tsx" - - "**.js" - - "**.jsx" - - "package.json" - - "pnpm-lock.yaml" - - "tsconfig.json" - - ".github/workflows/ci-node.yaml" - -permissions: - contents: read - -env: - NODE_VERSION: "22" - -jobs: - # --------------------------------------------------------------------------- - # Install dependencies (shared across jobs via cache) - # --------------------------------------------------------------------------- - install: - name: Install - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@v4 - - - uses: pnpm/action-setup@v4 - with: - version: latest - - - uses: actions/setup-node@v4 - with: - node-version: ${{ env.NODE_VERSION }} - cache: "pnpm" - - - name: Install dependencies - run: pnpm install --frozen-lockfile - - # --------------------------------------------------------------------------- - # Lint with ESLint - # --------------------------------------------------------------------------- - lint: - name: Lint - needs: install - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@v4 - - - uses: pnpm/action-setup@v4 - with: - version: latest - - - uses: actions/setup-node@v4 - with: - node-version: ${{ env.NODE_VERSION }} - cache: "pnpm" - - - run: pnpm install --frozen-lockfile - - run: pnpm lint - - # --------------------------------------------------------------------------- - # Type check with TypeScript compiler - # --------------------------------------------------------------------------- - type-check: - name: Type Check - needs: install - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@v4 - - - uses: pnpm/action-setup@v4 - with: - version: latest - - - uses: actions/setup-node@v4 - with: - node-version: ${{ env.NODE_VERSION }} - cache: "pnpm" - - - run: pnpm install --frozen-lockfile - - run: pnpm tsc --noEmit - - # --------------------------------------------------------------------------- - # Test with Vitest - # --------------------------------------------------------------------------- - test: - name: Test - needs: install - runs-on: ubuntu-latest - timeout-minutes: 15 - steps: - - uses: actions/checkout@v4 - - - uses: pnpm/action-setup@v4 - with: - version: latest - - - uses: actions/setup-node@v4 - with: - node-version: ${{ env.NODE_VERSION }} - cache: "pnpm" - - - run: pnpm install --frozen-lockfile - - - name: Run tests with coverage - run: pnpm vitest run --coverage --reporter=junit --outputFile=junit.xml - - - name: Upload coverage report - if: always() - uses: actions/upload-artifact@v4 - with: - name: coverage-report - path: coverage/ - retention-days: 7 - - - name: Upload test results - if: always() - uses: actions/upload-artifact@v4 - with: - name: test-results - path: junit.xml - retention-days: 7 - - # --------------------------------------------------------------------------- - # Build - # --------------------------------------------------------------------------- - build: - name: Build - needs: [lint, type-check, test] - runs-on: ubuntu-latest - timeout-minutes: 10 - steps: - - uses: actions/checkout@v4 - - - uses: pnpm/action-setup@v4 - with: - version: latest - - - uses: actions/setup-node@v4 - with: - node-version: ${{ env.NODE_VERSION }} - cache: "pnpm" - - - run: pnpm install --frozen-lockfile - - run: pnpm build - - - name: Upload build artifacts - uses: actions/upload-artifact@v4 - with: - name: build-output - path: dist/ - retention-days: 7 diff --git a/.claude/skills/devops/github-actions/templates/ci-python.yaml b/.claude/skills/devops/github-actions/templates/ci-python.yaml deleted file mode 100644 index 5895e3b..0000000 --- a/.claude/skills/devops/github-actions/templates/ci-python.yaml +++ /dev/null @@ -1,164 +0,0 @@ -# ============================================================================= -# Python CI Pipeline -# Runs: lint (ruff), type check (mypy), test (pytest), coverage upload -# ============================================================================= - -name: Python CI - -on: - push: - branches: [main] - paths: - - "**.py" - - "requirements*.txt" - - "pyproject.toml" - - ".github/workflows/ci-python.yaml" - pull_request: - branches: [main] - paths: - - "**.py" - - "requirements*.txt" - - "pyproject.toml" - - ".github/workflows/ci-python.yaml" - -permissions: - contents: read - -env: - PYTHON_VERSION: "3.12" - -jobs: - # --------------------------------------------------------------------------- - # Lint with Ruff - # --------------------------------------------------------------------------- - lint: - name: Lint - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@v4 - - - uses: actions/setup-python@v5 - with: - python-version: ${{ env.PYTHON_VERSION }} - - - name: Install ruff - run: pip install ruff - - - name: Ruff check (lint) - run: ruff check . - - - name: Ruff format (formatting) - run: ruff format --check . - - # --------------------------------------------------------------------------- - # Type check with mypy - # --------------------------------------------------------------------------- - type-check: - name: Type Check - runs-on: ubuntu-latest - timeout-minutes: 10 - steps: - - uses: actions/checkout@v4 - - - uses: actions/setup-python@v5 - with: - python-version: ${{ env.PYTHON_VERSION }} - - - name: Cache pip dependencies - uses: actions/cache@v4 - with: - path: ~/.cache/pip - key: ${{ runner.os }}-pip-${{ hashFiles('requirements*.txt') }} - restore-keys: | - ${{ runner.os }}-pip- - - - name: Install dependencies - run: | - pip install --upgrade pip - pip install -r requirements.txt - pip install mypy - - - name: Run mypy - run: mypy src/ --ignore-missing-imports - - # --------------------------------------------------------------------------- - # Test with pytest - # --------------------------------------------------------------------------- - test: - name: Test - runs-on: ubuntu-latest - timeout-minutes: 15 - - # Uncomment to add a PostgreSQL service for integration tests. - # services: - # postgres: - # image: postgres:17-alpine - # env: - # POSTGRES_USER: test - # POSTGRES_PASSWORD: test - # POSTGRES_DB: test_db - # ports: - # - 5432:5432 - # options: >- - # --health-cmd pg_isready - # --health-interval 10s - # --health-timeout 5s - # --health-retries 5 - - steps: - - uses: actions/checkout@v4 - - - uses: actions/setup-python@v5 - with: - python-version: ${{ env.PYTHON_VERSION }} - - - name: Cache pip dependencies - uses: actions/cache@v4 - with: - path: ~/.cache/pip - key: ${{ runner.os }}-pip-${{ hashFiles('requirements*.txt') }} - restore-keys: | - ${{ runner.os }}-pip- - - - name: Install dependencies - run: | - pip install --upgrade pip - pip install -r requirements.txt - pip install -r requirements-dev.txt - - - name: Run tests with coverage - run: | - pytest \ - --cov=src \ - --cov-report=xml:coverage.xml \ - --cov-report=term-missing \ - --junitxml=junit.xml \ - -v - env: - PYTHONPATH: ${{ github.workspace }} - # DATABASE_URL: postgresql://test:test@localhost:5432/test_db - - - name: Upload coverage report - if: always() - uses: actions/upload-artifact@v4 - with: - name: coverage-report - path: coverage.xml - retention-days: 7 - - - name: Upload test results - if: always() - uses: actions/upload-artifact@v4 - with: - name: test-results - path: junit.xml - retention-days: 7 - - # Uncomment to upload coverage to Codecov. - # - name: Upload to Codecov - # if: github.event_name == 'push' && github.ref == 'refs/heads/main' - # uses: codecov/codecov-action@v4 - # with: - # files: coverage.xml - # token: ${{ secrets.CODECOV_TOKEN }} diff --git a/.claude/skills/devops/references/cloudflare-workers.md b/.claude/skills/devops/references/cloudflare-workers.md new file mode 100644 index 0000000..6077fd9 --- /dev/null +++ b/.claude/skills/devops/references/cloudflare-workers.md @@ -0,0 +1,545 @@ +# DevOps — Cloudflare Workers Patterns + + +# Cloudflare Workers & Pages + +## Overview + +Edge-first deployment patterns for Cloudflare's platform. Covers Workers (compute), Pages (static + SSR), R2 (object storage), D1 (SQLite at edge), KV (key-value), Durable Objects (stateful), and Queues (async processing). Focused on the Python/TypeScript stack this kit targets. + +## When to Use +- Deploying APIs or full-stack apps to Cloudflare's edge network +- Building serverless functions with Workers +- Deploying Next.js or static sites via Cloudflare Pages +- Using D1 (edge SQLite), R2 (S3-compatible storage), or KV (low-latency reads) +- Implementing real-time coordination with Durable Objects +- Background job processing with Cloudflare Queues + +## When NOT to Use +- **Long-running compute** (> 30s CPU) — use traditional servers or containers +- **Heavy database workloads** — D1 is SQLite; use Postgres/Mongo for complex queries +- **GPU/ML inference** (unless using Workers AI) — use dedicated compute +- **Local-only development** — Workers run on V8 isolates, not Node.js + +--- + +## Quick Reference + +| I need... | Go to | +|-----------|-------| +| Worker project structure | § Project Structure below | +| Hono framework on Workers | § Hono Framework below | +| D1 database patterns | § D1 (Edge SQLite) below | +| R2 object storage | § R2 (Object Storage) below | +| KV key-value store | § KV below | +| Durable Objects | § Durable Objects below | +| Pages deployment (Next.js) | § Cloudflare Pages below | +| CI/CD with GitHub Actions | § CI/CD below | +| Wrangler config reference | See `wrangler-patterns.md` in this skill's directory | + +--- + +## Project Structure + +``` +my-worker/ +├── wrangler.toml # Wrangler config (bindings, routes, env) +├── src/ +│ ├── index.ts # Entry point (fetch handler) +│ ├── routes/ # Route handlers +│ ├── middleware/ # Auth, CORS, logging +│ ├── services/ # Business logic +│ └── types.ts # Env bindings type +├── migrations/ # D1 migrations +├── test/ # Vitest tests +└── package.json +``` + +### Entry point + +```typescript +// src/index.ts +export default { + async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise { + const url = new URL(request.url); + + if (url.pathname === '/health') { + return Response.json({ status: 'ok' }); + } + + // Route to handlers... + return new Response('Not found', { status: 404 }); + }, +} satisfies ExportedHandler; +``` + +### Type-safe bindings + +```typescript +// src/types.ts +export interface Env { + DB: D1Database; + BUCKET: R2Bucket; + CACHE: KVNamespace; + API_KEY: string; + ENVIRONMENT: 'development' | 'staging' | 'production'; +} +``` + +--- + +## Hono Framework (Recommended) + +Hono is the de facto framework for Workers — ultralight (~14KB), type-safe, and built for edge runtimes. + +```typescript +// src/index.ts +import { Hono } from 'hono'; +import { cors } from 'hono/cors'; +import { logger } from 'hono/logger'; +import { HTTPException } from 'hono/http-exception'; +import { zValidator } from '@hono/zod-validator'; +import { z } from 'zod'; + +type Bindings = { + DB: D1Database; + BUCKET: R2Bucket; + API_KEY: string; +}; + +const app = new Hono<{ Bindings: Bindings }>(); + +app.use('*', logger()); +app.use('*', cors({ origin: ['https://app.example.com'], credentials: true })); + +// Health check +app.get('/health', (c) => c.json({ status: 'ok' })); + +// Validated endpoint +const createUserSchema = z.object({ + email: z.string().email().max(254), + name: z.string().min(1).max(100), +}); + +app.post('/v1/users', zValidator('json', createUserSchema), async (c) => { + const { email, name } = c.req.valid('json'); + const result = await c.env.DB + .prepare('INSERT INTO users (id, email, name) VALUES (?, ?, ?) RETURNING *') + .bind(crypto.randomUUID(), email, name) + .first(); + return c.json(result, 201); +}); + +// Error handling — RFC 9457 Problem Details +app.onError((err, c) => { + if (err instanceof HTTPException) { + return c.json({ + type: `https://api.example.com/problems/${err.status}`, + title: err.message, + status: err.status, + }, err.status); + } + console.error(err); + return c.json({ + type: 'https://api.example.com/problems/internal-error', + title: 'Internal server error', + status: 500, + }, 500); +}); + +export default app; +``` + +--- + +## D1 (Edge SQLite) + +Cloudflare's serverless SQL database. SQLite at the edge with automatic replication. + +### Migrations + +```bash +# Create migration +npx wrangler d1 migrations create my-db create-users + +# Apply locally +npx wrangler d1 migrations apply my-db --local + +# Apply to production +npx wrangler d1 migrations apply my-db --remote +``` + +```sql +-- migrations/0001_create-users.sql +CREATE TABLE IF NOT EXISTS users ( + id TEXT PRIMARY KEY, + email TEXT UNIQUE NOT NULL, + name TEXT NOT NULL, + role TEXT DEFAULT 'member' CHECK(role IN ('admin', 'member', 'viewer')), + created_at TEXT DEFAULT (datetime('now')), + updated_at TEXT DEFAULT (datetime('now')) +); + +CREATE INDEX idx_users_email ON users(email); +``` + +### Querying with prepared statements + +```typescript +// Always use prepared statements — never concatenate SQL +async function getUser(db: D1Database, id: string) { + return db.prepare('SELECT * FROM users WHERE id = ?').bind(id).first(); +} + +async function listUsers(db: D1Database, cursor?: string, limit = 20) { + const stmt = cursor + ? db.prepare('SELECT * FROM users WHERE id > ? ORDER BY id LIMIT ?').bind(cursor, limit) + : db.prepare('SELECT * FROM users ORDER BY id LIMIT ?').bind(limit); + return stmt.all(); +} + +// Batch multiple statements in a transaction +async function transferCredits(db: D1Database, from: string, to: string, amount: number) { + const results = await db.batch([ + db.prepare('UPDATE accounts SET balance = balance - ? WHERE id = ?').bind(amount, from), + db.prepare('UPDATE accounts SET balance = balance + ? WHERE id = ?').bind(amount, to), + ]); + return results; +} +``` + +### D1 limitations to know + +- **No JOINs across databases** — one D1 database per binding +- **5MB max row size**, 10GB max database +- **Read replicas are automatic** but writes go to a single leader +- **No stored procedures / triggers** — SQLite subset +- **Prepared statements are mandatory** — `db.exec()` with raw SQL is for migrations only + +--- + +## R2 (Object Storage) + +S3-compatible object storage without egress fees. + +```typescript +// Upload +app.put('/v1/files/:key', async (c) => { + const key = c.req.param('key'); + const body = await c.req.arrayBuffer(); + const contentType = c.req.header('Content-Type') ?? 'application/octet-stream'; + + await c.env.BUCKET.put(key, body, { + httpMetadata: { contentType }, + customMetadata: { uploadedBy: c.get('userId') }, + }); + + return c.json({ key, size: body.byteLength }, 201); +}); + +// Download +app.get('/v1/files/:key', async (c) => { + const obj = await c.env.BUCKET.get(c.req.param('key')); + if (!obj) return c.json({ error: 'Not found' }, 404); + + return new Response(obj.body, { + headers: { + 'Content-Type': obj.httpMetadata?.contentType ?? 'application/octet-stream', + 'ETag': obj.etag, + }, + }); +}); + +// List with prefix +app.get('/v1/files', async (c) => { + const prefix = c.req.query('prefix') ?? ''; + const listed = await c.env.BUCKET.list({ prefix, limit: 100 }); + return c.json({ objects: listed.objects.map((o) => ({ key: o.key, size: o.size })) }); +}); +``` + +### Presigned URLs for direct upload + +```typescript +// Generate a presigned URL so clients upload directly to R2 +app.post('/v1/upload-url', async (c) => { + const key = `uploads/${crypto.randomUUID()}`; + // Use the S3-compatible API for presigned URLs + // Requires R2 API token with write access + return c.json({ key, uploadUrl: `https://${ACCOUNT_ID}.r2.cloudflarestorage.com/${BUCKET_NAME}/${key}` }); +}); +``` + +--- + +## KV (Key-Value Store) + +Global low-latency reads (~10ms worldwide), eventually consistent writes. + +```typescript +// Set with TTL +await c.env.CACHE.put('session:abc123', JSON.stringify(sessionData), { + expirationTtl: 3600, // 1 hour +}); + +// Get with type safety +const raw = await c.env.CACHE.get('session:abc123'); +const session = raw ? JSON.parse(raw) as SessionData : null; + +// List keys by prefix +const keys = await c.env.CACHE.list({ prefix: 'session:' }); + +// Delete +await c.env.CACHE.delete('session:abc123'); +``` + +**Use KV for:** session tokens, feature flags, cached API responses, configuration. **Not for:** frequently updated counters, multi-key transactions (use Durable Objects). + +--- + +## Durable Objects + +Stateful, single-instance coordination. Each Durable Object has a unique ID and runs in exactly one location. + +```typescript +// src/counter.ts +export class Counter implements DurableObject { + private count = 0; + + constructor(private state: DurableObjectState, private env: Env) {} + + async fetch(request: Request): Promise { + const url = new URL(request.url); + + if (url.pathname === '/increment') { + this.count++; + await this.state.storage.put('count', this.count); + return Response.json({ count: this.count }); + } + + this.count = (await this.state.storage.get('count')) ?? 0; + return Response.json({ count: this.count }); + } +} + +// In the Worker, route to the Durable Object: +app.post('/v1/counters/:name/increment', async (c) => { + const id = c.env.COUNTER.idFromName(c.req.param('name')); + const stub = c.env.COUNTER.get(id); + const res = await stub.fetch(new Request('https://dummy/increment')); + return c.json(await res.json()); +}); +``` + +**Use Durable Objects for:** rate limiting, WebSocket rooms, collaborative editing, distributed locks, shopping carts. **Not for:** read-heavy caching (use KV). + +--- + +## Cloudflare Pages + +### Next.js on Pages + +```bash +# Deploy Next.js to Cloudflare Pages +npx wrangler pages deploy .next --project-name=my-app +``` + +Use `@cloudflare/next-on-pages` for full App Router + Server Components support: + +```bash +pnpm add @cloudflare/next-on-pages +``` + +```typescript +// next.config.ts +import { setupDevPlatform } from '@cloudflare/next-on-pages/next-dev'; + +if (process.env.NODE_ENV === 'development') { + await setupDevPlatform(); +} + +const nextConfig = { /* ... */ }; +export default nextConfig; +``` + +### Static site on Pages + +```bash +# Build and deploy +pnpm build +npx wrangler pages deploy dist/ --project-name=my-site +``` + +Pages auto-deploys from GitHub: connect your repo in the Cloudflare dashboard, set the build command and output directory. Preview deploys on every PR. + +--- + +## Wrangler Config + +```toml +# wrangler.toml +name = "my-api" +main = "src/index.ts" +compatibility_date = "2026-01-01" +compatibility_flags = ["nodejs_compat"] + +[vars] +ENVIRONMENT = "production" + +# D1 database +[[d1_databases]] +binding = "DB" +database_name = "my-db" +database_id = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" + +# R2 bucket +[[r2_buckets]] +binding = "BUCKET" +bucket_name = "my-bucket" + +# KV namespace +[[kv_namespaces]] +binding = "CACHE" +id = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" + +# Durable Object +[[durable_objects.bindings]] +name = "COUNTER" +class_name = "Counter" + +[[migrations]] +tag = "v1" +new_classes = ["Counter"] + +# Environment overrides +[env.staging] +vars = { ENVIRONMENT = "staging" } + +[env.staging.d1_databases] +binding = "DB" +database_name = "my-db-staging" +database_id = "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy" +``` + +**`compatibility_date`** pins your Worker to a specific runtime version. Always set it to a recent date and update periodically. **`nodejs_compat`** enables Node.js built-in APIs (Buffer, crypto, streams) — required for most npm packages. + +--- + +## CI/CD + +### GitHub Actions deploy + +```yaml +# .github/workflows/deploy.yml +name: Deploy Worker +on: + push: + branches: [main] + pull_request: + +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: { node-version: '20' } + - run: pnpm install + + - name: Run tests + run: pnpm test + + - name: Apply D1 migrations (production) + if: github.ref == 'refs/heads/main' + run: npx wrangler d1 migrations apply my-db --remote + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CF_API_TOKEN }} + + - name: Deploy to staging (PR) + if: github.event_name == 'pull_request' + run: npx wrangler deploy --env staging + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CF_API_TOKEN }} + + - name: Deploy to production + if: github.ref == 'refs/heads/main' + run: npx wrangler deploy + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CF_API_TOKEN }} +``` + +### Local development + +```bash +# Start local dev server with all bindings (D1, R2, KV, DO) +npx wrangler dev + +# With local D1 persistence +npx wrangler dev --persist-to .wrangler/state +``` + +`wrangler dev` uses Miniflare under the hood — a local simulator for all Cloudflare primitives. Test against real bindings locally before deploying. + +--- + +## Testing + +Use **Vitest + Miniflare** (via `@cloudflare/vitest-pool-workers`): + +```typescript +// vitest.config.ts +import { defineWorkersConfig } from '@cloudflare/vitest-pool-workers/config'; + +export default defineWorkersConfig({ + test: { + poolOptions: { + workers: { + wrangler: { configPath: './wrangler.toml' }, + }, + }, + }, +}); +``` + +```typescript +// test/index.spec.ts +import { env, createExecutionContext, waitOnExecutionContext } from 'cloudflare:test'; +import { describe, it, expect } from 'vitest'; +import worker from '../src/index'; + +describe('Worker', () => { + it('returns health check', async () => { + const request = new Request('http://localhost/health'); + const ctx = createExecutionContext(); + const response = await worker.fetch(request, env, ctx); + await waitOnExecutionContext(ctx); + + expect(response.status).toBe(200); + const body = await response.json(); + expect(body).toEqual({ status: 'ok' }); + }); +}); +``` + +--- + +## Common Pitfalls + +1. **Using Node.js APIs without `nodejs_compat`.** Workers run on V8, not Node.js. Without the flag, `Buffer`, `crypto`, `process` are undefined. +2. **Blocking the event loop.** Workers have strict CPU time limits (10ms free, 30s paid). Heavy computation blocks all concurrent requests. Use `ctx.waitUntil()` for background work. +3. **Ignoring D1's eventually consistent reads.** Writes go to the leader; reads from replicas may lag by seconds. Design for eventual consistency. +4. **Using KV for frequently updated data.** KV is eventually consistent with ~60s propagation. Use Durable Objects for strong consistency. +5. **Not setting `compatibility_date`.** Without it, you get the oldest runtime behavior. Always pin to a recent date. +6. **Forgetting `ctx.waitUntil()`.** Background work (logging, analytics) must be wrapped in `waitUntil()` or it gets killed when the response is sent. +7. **Large Worker bundles.** Workers have a 10MB compressed limit (free: 1MB). Tree-shake aggressively; avoid heavy npm packages. +8. **Not testing locally with Miniflare.** `wrangler dev` simulates all bindings locally. Deploying untested changes to edge = debugging in production. + +--- + +## Related Skills + +- `openapi` — API design (Workers APIs benefit from the same conventions) +- `docker` — alternative deployment model (containers vs edge) +- `github-actions` — CI/CD pipeline for deploying Workers +- `typescript` — TypeScript patterns (Workers are TypeScript-first) +- `vitest` — testing Workers with Miniflare pool diff --git a/.claude/skills/devops/docker/SKILL.md b/.claude/skills/devops/references/docker.md similarity index 95% rename from .claude/skills/devops/docker/SKILL.md rename to .claude/skills/devops/references/docker.md index e56d2b4..6fcc7f9 100644 --- a/.claude/skills/devops/docker/SKILL.md +++ b/.claude/skills/devops/references/docker.md @@ -1,8 +1,5 @@ ---- -name: docker -description: > - Use this skill whenever containerizing applications, writing Dockerfiles, configuring Docker Compose, or optimizing container images. Trigger on keywords like Docker, Dockerfile, container, docker-compose, multi-stage build, image, or container registry. Also applies when setting up local development environments with containers, debugging container networking, or preparing applications for container-based deployment in CI/CD pipelines. ---- +# DevOps — Docker Patterns + # Docker @@ -654,6 +651,6 @@ services: ## Related Skills -- `devops/github-actions` - CI/CD workflows for building and deploying Docker containers -- `security/owasp` - Security best practices for container hardening and vulnerability scanning -- `patterns/logging` — Container logging and log aggregation +- `github-actions` - CI/CD workflows for building and deploying Docker containers +- `owasp` - Security best practices for container hardening and vulnerability scanning +- `logging` — Container logging and log aggregation diff --git a/.claude/skills/devops/github-actions/SKILL.md b/.claude/skills/devops/references/github-actions.md similarity index 96% rename from .claude/skills/devops/github-actions/SKILL.md rename to .claude/skills/devops/references/github-actions.md index 8de58cb..7d6e0f3 100644 --- a/.claude/skills/devops/github-actions/SKILL.md +++ b/.claude/skills/devops/references/github-actions.md @@ -1,8 +1,5 @@ ---- -name: github-actions -description: > - Use this skill whenever setting up or modifying GitHub Actions CI/CD workflows, automating tests, builds, or deployments on GitHub. Trigger on keywords like GitHub Actions, workflow YAML, CI/CD pipeline, actions/checkout, matrix builds, workflow_dispatch, or .github/workflows. Also applies when configuring caching in workflows, managing GitHub secrets, or troubleshooting failed workflow runs. ---- +# DevOps — GitHub Actions Patterns + # GitHub Actions @@ -799,6 +796,6 @@ jobs: ## Related Skills -- `devops/docker` - Container patterns for building and deploying Dockerized applications in workflows -- `testing/pytest` - Python test configuration for CI pipeline integration -- `testing/vitest` - TypeScript/JavaScript test configuration for CI pipeline integration +- `docker` - Container patterns for building and deploying Dockerized applications in workflows +- `pytest` - Python test configuration for CI pipeline integration +- `vitest` - TypeScript/JavaScript test configuration for CI pipeline integration diff --git a/.claude/skills/methodology/dispatching-parallel-agents/SKILL.md b/.claude/skills/dispatching-parallel-agents/SKILL.md similarity index 63% rename from .claude/skills/methodology/dispatching-parallel-agents/SKILL.md rename to .claude/skills/dispatching-parallel-agents/SKILL.md index 4b744a8..ef23278 100644 --- a/.claude/skills/methodology/dispatching-parallel-agents/SKILL.md +++ b/.claude/skills/dispatching-parallel-agents/SKILL.md @@ -1,7 +1,7 @@ --- name: dispatching-parallel-agents description: > - Trigger this skill when facing 3 or more independent failures across different domains, when multiple subsystems are broken with no shared state, or when test failures span unrelated modules. Use whenever you see independent bugs in auth, cart, user, or other separate domains that can be fixed concurrently. Activate aggressively for any scenario where parallel work would reduce total resolution time without creating merge conflicts. + Use when facing 3 or more independent failures across different domains, when multiple subsystems are broken with no shared state, or when test failures span unrelated modules. Also activate whenever you see independent bugs in auth, cart, user, or other separate domains that can be fixed concurrently. Activate aggressively for any scenario where parallel work would reduce total resolution time without creating merge conflicts. --- # Dispatching Parallel Agents @@ -229,6 +229,69 @@ git merge agent-3-user-fixes --- +## Example: Full-Stack Feature Dispatch + +A real-world example dispatching 3 agents for a new "orders" feature: + +### Independence check + +| | Agent 1 (Backend) | Agent 2 (Frontend) | Agent 3 (Database) | +|---|---|---|---| +| **Files** | `src/api/orders.py`, `tests/test_orders.py` | `src/components/order-form.tsx`, `*.test.tsx` | `migrations/003_orders.sql`, `tests/test_migration.py` | +| **Test suite** | `pytest tests/test_orders.py` | `npm test -- order-form` | `pytest tests/test_migration.py` | +| **Shared state?** | No | No | No | + +All three touch different files and different test suites — safe to parallelize. + +### Agent 1 — Backend (FastAPI) + +```markdown +## Task: Implement POST /api/orders with validation + +**Context**: FastAPI + SQLAlchemy async + Pydantic v2 +**Files**: src/api/orders.py, src/schemas/order.py, tests/test_orders.py +**Constraints**: Depends(get_db), return 201, RFC 9457 errors +**Verify**: pytest tests/test_orders.py -v +``` + +### Agent 2 — Frontend (React/Next.js) + +```markdown +## Task: Build OrderForm component with validation + +**Context**: Next.js App Router + react-hook-form + Zod + shadcn/ui +**Files**: src/components/order-form.tsx, src/components/order-form.test.tsx +**Constraints**: 'use client', Zod schema, accessible form fields +**Verify**: npx vitest run src/components/order-form.test.tsx +``` + +### Agent 3 — Database (PostgreSQL) + +```markdown +## Task: Create orders table migration + +**Context**: Alembic migrations, PostgreSQL +**Files**: migrations/003_create_orders.sql, tests/test_orders_migration.py +**Constraints**: Include indexes on user_id and created_at, add foreign key to users +**Verify**: pytest tests/test_orders_migration.py -v +``` + +### Integration after all 3 complete + +```bash +# 1. Run each agent's test suite to confirm +pytest tests/test_orders.py tests/test_orders_migration.py -v +npx vitest run src/components/order-form.test.tsx + +# 2. Run full test suite for regressions +pytest -v && npm test + +# 3. Verify no file conflicts +git diff --name-only # should show no overlapping files between agents +``` + +--- + ## Conflict Resolution If conflicts detected: @@ -262,5 +325,5 @@ After parallel completion: ## Related Skills -- `methodology/executing-plans` - Use executing-plans when tasks are sequential; use dispatching-parallel-agents when tasks are independent and can run concurrently -- `methodology/writing-plans` - Write a plan first to identify which tasks are independent before dispatching parallel agents +- `executing-plans` - Use executing-plans when tasks are sequential; use dispatching-parallel-agents when tasks are independent and can run concurrently +- `writing-plans` - Write a plan first to identify which tasks are independent before dispatching parallel agents diff --git a/.claude/skills/methodology/dispatching-parallel-agents/references/parallelization-patterns.md b/.claude/skills/dispatching-parallel-agents/references/parallelization-patterns.md similarity index 100% rename from .claude/skills/methodology/dispatching-parallel-agents/references/parallelization-patterns.md rename to .claude/skills/dispatching-parallel-agents/references/parallelization-patterns.md diff --git a/.claude/skills/error-handling/SKILL.md b/.claude/skills/error-handling/SKILL.md new file mode 100644 index 0000000..d1d273c --- /dev/null +++ b/.claude/skills/error-handling/SKILL.md @@ -0,0 +1,90 @@ +--- +name: error-handling +description: > + Use when writing try/catch blocks, creating custom error classes, implementing retry logic, designing error boundaries in React, building API error responses, or handling failures gracefully. Also activate for any code dealing with exceptions, error propagation, graceful degradation, or fault tolerance. +--- + +# Error Handling Patterns + +## When to Use + +- Building API endpoints that must return consistent error responses +- Creating custom exception hierarchies for a domain model +- Implementing retry logic for unreliable network calls or external services +- Designing React error boundaries for component-level fault isolation +- Wrapping third-party libraries that throw unpredictable errors +- Converting between error representations at architectural boundaries (e.g., domain errors to HTTP errors) +- Adopting the Result pattern to avoid exceptions for expected failure paths + +## When NOT to Use + +- Simple one-off scripts or throwaway prototypes where unhandled crashes are acceptable +- Configuration files, static data, or declarative markup with no runtime logic +- Pure data transformation functions where invalid input should be prevented by types, not caught at runtime + +--- + +## Quick Reference + +| Pattern | Description | +|---------|-------------| +| Custom Error Classes | Domain-specific error hierarchy with error codes, messages, and detail metadata | +| Error Boundaries (React) | Component-level fault isolation using `react-error-boundary` or class-based boundaries | +| Retry with Backoff | Exponential backoff + jitter decorator/wrapper for transient failures | +| Circuit Breaker | Short-circuit calls to unhealthy dependencies, fall back to degraded state | +| Feature-Flag Degradation | Graceful UI/service degradation controlled by feature flags | +| API Error Responses | Consistent RFC 7807 Problem Details payloads with global exception handlers | +| Error Logging | Structured context (request ID, error code, stack trace) for observability | +| Result Pattern | Discriminated union / Result type for expected failure paths without exceptions | + +## Language References + +See `references/python-patterns.md` for Python examples. + +See `references/typescript-patterns.md` for TypeScript/React examples. + +--- + +## Best Practices + +1. **Catch specific exceptions, not bare `except` or `catch`.** A catch-all hides bugs. Catch only the errors you know how to handle and let everything else propagate. + +2. **Translate errors at architectural boundaries.** A database `IntegrityError` should become a domain `DuplicateEntryError` at the repository layer, then an HTTP 409 at the API layer. Each layer speaks its own error language. + +3. **Preserve the original cause.** Always chain the original exception (`raise X from original` in Python, `{ cause }` in TypeScript) so the root cause is visible in logs and debuggers. + +4. **Fail fast, recover high.** Detect errors as early as possible (validate inputs at the boundary) but handle them at the highest level that has enough context to decide what to do (e.g., return an HTTP response, show a fallback UI). + +5. **Never swallow errors silently.** An empty `except: pass` or `catch {}` is almost always a bug. At minimum, log the error. If you intentionally ignore it, leave a comment explaining why. + +6. **Use the Result pattern for expected failures.** When a function can legitimately fail (parsing, validation, lookups), return a Result instead of throwing. Reserve exceptions for truly unexpected situations. + +7. **Make errors actionable.** Every error message should help the reader fix the problem. Include what happened, what was expected, and what the caller can do about it. `"User not found"` is worse than `"User with id '123' not found. Verify the id and check that the user has not been deleted."`. + +8. **Test the error paths.** Write explicit tests for every error branch. Verify the error type, message, and status code. Error paths that are never tested are error paths that will break in production. + +--- + +## Common Pitfalls + +1. **Catching too broadly.** Using `except Exception` or `catch (e: any)` silences programming errors like `TypeError` or `ReferenceError` that should crash loudly during development. + +2. **Logging and re-throwing without deduplication.** If every layer logs the same error, you get five log entries for one failure. Log at the outermost handler and let inner layers propagate. + +3. **Returning error data in the wrong shape.** Mixing `{ error: "..." }`, `{ message: "..." }`, and `{ errors: [...] }` across endpoints forces every client to handle multiple formats. Pick one shape and enforce it globally. + +4. **Leaking internal details to clients.** Stack traces, database table names, and file paths in API responses are a security risk. Sanitize errors before they leave the server. + +5. **Retrying non-idempotent operations.** Retrying a `POST /orders` that partially succeeded can create duplicate orders. Only retry operations that are safe to repeat, or use idempotency keys. + +6. **Ignoring async error boundaries.** In React, error boundaries do not catch errors inside event handlers or async callbacks. Use try/catch inside `onClick`, `useEffect` cleanup, and promise chains separately. + +--- + +## Related Skills + +- `logging` - Structured logging setup and conventions +- `api-client` - HTTP client wrappers with built-in error handling +- `owasp` - Preventing information leakage through error messages +- `python` - Python exception syntax and idioms +- `typescript` - TypeScript error types and narrowing diff --git a/.claude/skills/patterns/error-handling/references/error-taxonomy.md b/.claude/skills/error-handling/references/error-taxonomy.md similarity index 100% rename from .claude/skills/patterns/error-handling/references/error-taxonomy.md rename to .claude/skills/error-handling/references/error-taxonomy.md diff --git a/.claude/skills/error-handling/references/python-patterns.md b/.claude/skills/error-handling/references/python-patterns.md new file mode 100644 index 0000000..191c8cc --- /dev/null +++ b/.claude/skills/error-handling/references/python-patterns.md @@ -0,0 +1,388 @@ +# Error Handling — Python Patterns + +## 1. Custom Error Classes + +Define a hierarchy of domain-specific errors so callers can catch at the right granularity. + +```python +from enum import Enum + + +class ErrorCode(str, Enum): + """Central registry of machine-readable error codes.""" + NOT_FOUND = "NOT_FOUND" + VALIDATION_FAILED = "VALIDATION_FAILED" + DUPLICATE_ENTRY = "DUPLICATE_ENTRY" + UNAUTHORIZED = "UNAUTHORIZED" + RATE_LIMITED = "RATE_LIMITED" + EXTERNAL_SERVICE = "EXTERNAL_SERVICE" + INTERNAL = "INTERNAL" + + +class AppError(Exception): + """Base error for the entire application. + + All domain errors inherit from this so a single except clause + can catch everything the application intentionally raises. + """ + + def __init__( + self, + message: str, + code: ErrorCode = ErrorCode.INTERNAL, + *, + details: dict | None = None, + cause: Exception | None = None, + ) -> None: + super().__init__(message) + self.code = code + self.details = details or {} + if cause: + self.__cause__ = cause + + def to_dict(self) -> dict: + return { + "error": self.code.value, + "message": str(self), + "details": self.details, + } + + +class NotFoundError(AppError): + def __init__(self, resource: str, identifier: str) -> None: + super().__init__( + f"{resource} with id '{identifier}' not found", + code=ErrorCode.NOT_FOUND, + details={"resource": resource, "id": identifier}, + ) + + +class ValidationError(AppError): + def __init__(self, field: str, reason: str) -> None: + super().__init__( + f"Validation failed for '{field}': {reason}", + code=ErrorCode.VALIDATION_FAILED, + details={"field": field, "reason": reason}, + ) + + +class ExternalServiceError(AppError): + def __init__(self, service: str, cause: Exception) -> None: + super().__init__( + f"External service '{service}' failed: {cause}", + code=ErrorCode.EXTERNAL_SERVICE, + cause=cause, + details={"service": service}, + ) +``` + +## 2. Retry Pattern + +Retry transient failures with exponential backoff and jitter to avoid thundering herd. + +```python +import asyncio +import random +import logging +from functools import wraps +from typing import TypeVar, Callable, Awaitable + +logger = logging.getLogger(__name__) + +T = TypeVar("T") + + +def retry( + max_attempts: int = 3, + base_delay: float = 1.0, + max_delay: float = 30.0, + retryable: tuple[type[Exception], ...] = (Exception,), +) -> Callable: + """Retry decorator with exponential backoff and full jitter. + + Args: + max_attempts: Total number of attempts including the first call. + base_delay: Initial delay in seconds before the first retry. + max_delay: Upper bound on the computed delay. + retryable: Exception types eligible for retry. + """ + + def decorator(fn: Callable[..., Awaitable[T]]) -> Callable[..., Awaitable[T]]: + @wraps(fn) + async def wrapper(*args, **kwargs) -> T: + last_exc: Exception | None = None + for attempt in range(1, max_attempts + 1): + try: + return await fn(*args, **kwargs) + except retryable as exc: + last_exc = exc + if attempt == max_attempts: + break + delay = min(base_delay * (2 ** (attempt - 1)), max_delay) + jitter = random.uniform(0, delay) + logger.warning( + "Attempt %d/%d failed (%s), retrying in %.2fs", + attempt, + max_attempts, + exc, + jitter, + ) + await asyncio.sleep(jitter) + raise last_exc # type: ignore[misc] + + return wrapper + + return decorator + + +# Usage +@retry(max_attempts=3, base_delay=0.5, retryable=(ConnectionError, TimeoutError)) +async def fetch_remote_config(url: str) -> dict: + async with httpx.AsyncClient(timeout=5) as client: + resp = await client.get(url) + resp.raise_for_status() + return resp.json() +``` + +## 3. Graceful Degradation — Circuit Breaker + +When a dependency fails, fall back to a degraded but functional state instead of crashing. + +```python +import time +from enum import Enum + + +class CircuitState(Enum): + CLOSED = "closed" # normal operation + OPEN = "open" # failing, reject immediately + HALF_OPEN = "half_open" # testing recovery + + +class CircuitBreaker: + """Prevents cascading failures by short-circuiting calls to an unhealthy dependency.""" + + def __init__( + self, + failure_threshold: int = 5, + recovery_timeout: float = 30.0, + ) -> None: + self.failure_threshold = failure_threshold + self.recovery_timeout = recovery_timeout + self.state = CircuitState.CLOSED + self.failure_count = 0 + self.last_failure_time = 0.0 + + def _trip(self) -> None: + self.state = CircuitState.OPEN + self.last_failure_time = time.monotonic() + + def _reset(self) -> None: + self.state = CircuitState.CLOSED + self.failure_count = 0 + + async def call(self, fn, *args, fallback=None, **kwargs): + if self.state == CircuitState.OPEN: + if time.monotonic() - self.last_failure_time > self.recovery_timeout: + self.state = CircuitState.HALF_OPEN + else: + if fallback is not None: + return fallback() if callable(fallback) else fallback + raise ExternalServiceError("circuit-breaker", RuntimeError("Circuit open")) + + try: + result = await fn(*args, **kwargs) + if self.state == CircuitState.HALF_OPEN: + self._reset() + return result + except Exception as exc: + self.failure_count += 1 + if self.failure_count >= self.failure_threshold: + self._trip() + raise + + +# Usage +recommendations_circuit = CircuitBreaker(failure_threshold=3, recovery_timeout=60) + +async def get_recommendations(user_id: str) -> list[dict]: + return await recommendations_circuit.call( + recommendation_service.fetch, + user_id, + fallback=lambda: [], # empty list when service is down + ) +``` + +## 4. API Error Responses (FastAPI) + +Return consistent, machine-readable error payloads following RFC 7807 Problem Details. + +```python +from fastapi import FastAPI, Request +from fastapi.responses import JSONResponse +from starlette.status import ( + HTTP_400_BAD_REQUEST, + HTTP_404_NOT_FOUND, + HTTP_429_TOO_MANY_REQUESTS, + HTTP_500_INTERNAL_SERVER_ERROR, +) + +app = FastAPI() + +# Map domain error codes to HTTP status codes +STATUS_MAP: dict[ErrorCode, int] = { + ErrorCode.NOT_FOUND: HTTP_404_NOT_FOUND, + ErrorCode.VALIDATION_FAILED: HTTP_400_BAD_REQUEST, + ErrorCode.DUPLICATE_ENTRY: 409, + ErrorCode.UNAUTHORIZED: 401, + ErrorCode.RATE_LIMITED: HTTP_429_TOO_MANY_REQUESTS, + ErrorCode.EXTERNAL_SERVICE: 502, + ErrorCode.INTERNAL: HTTP_500_INTERNAL_SERVER_ERROR, +} + + +@app.exception_handler(AppError) +async def app_error_handler(request: Request, exc: AppError) -> JSONResponse: + status = STATUS_MAP.get(exc.code, 500) + return JSONResponse( + status_code=status, + content={ + "type": f"https://docs.example.com/errors/{exc.code.value.lower()}", + "title": exc.code.value.replace("_", " ").title(), + "status": status, + "detail": str(exc), + **exc.details, + }, + ) + + +@app.exception_handler(Exception) +async def unhandled_error_handler(request: Request, exc: Exception) -> JSONResponse: + # Never leak internal details to the client + logger.exception("Unhandled exception on %s %s", request.method, request.url.path) + return JSONResponse( + status_code=500, + content={ + "type": "https://docs.example.com/errors/internal", + "title": "Internal Server Error", + "status": 500, + "detail": "An unexpected error occurred.", + }, + ) +``` + +## 5. Error Logging Integration + +Attach structured context to errors so they are searchable and actionable in observability tools. + +```python +import logging +import traceback +from contextvars import ContextVar + +request_id_var: ContextVar[str] = ContextVar("request_id", default="unknown") + + +class StructuredErrorLogger: + """Wraps the standard logger to attach error context automatically.""" + + def __init__(self, name: str) -> None: + self.logger = logging.getLogger(name) + + def error( + self, + msg: str, + *, + exc: Exception | None = None, + extra: dict | None = None, + ) -> None: + context = { + "request_id": request_id_var.get(), + **(extra or {}), + } + + if exc is not None: + context["error_type"] = type(exc).__name__ + context["error_message"] = str(exc) + context["stacktrace"] = traceback.format_exception(exc) + + if isinstance(exc, AppError): + context["error_code"] = exc.code.value + context["error_details"] = exc.details + + self.logger.error(msg, extra={"structured": context}, exc_info=exc) + + +# Usage in a FastAPI middleware +from starlette.middleware.base import BaseHTTPMiddleware +import uuid + +log = StructuredErrorLogger(__name__) + + +class ErrorLoggingMiddleware(BaseHTTPMiddleware): + async def dispatch(self, request, call_next): + rid = request.headers.get("x-request-id", str(uuid.uuid4())) + request_id_var.set(rid) + try: + response = await call_next(request) + return response + except Exception as exc: + log.error( + "Request failed", + exc=exc, + extra={"method": request.method, "path": request.url.path}, + ) + raise +``` + +## 6. Result Pattern + +Use a Result type for operations where failure is an expected outcome. Avoids exception overhead and makes the failure path explicit in the type signature. + +```python +# pip install result +from result import Ok, Err, Result + + +def parse_age(value: str) -> Result[int, str]: + """Parse a string to a valid age. Returns Err for invalid input.""" + try: + age = int(value) + except ValueError: + return Err(f"'{value}' is not a number") + + if age < 0 or age > 150: + return Err(f"Age {age} is out of valid range (0-150)") + + return Ok(age) + + +def validate_registration(data: dict) -> Result[dict, list[str]]: + """Validate all fields, collecting every error instead of failing on the first.""" + errors: list[str] = [] + + match parse_age(data.get("age", "")): + case Ok(age): + data["age"] = age + case Err(msg): + errors.append(msg) + + name = data.get("name", "").strip() + if not name: + errors.append("Name is required") + if len(name) > 100: + errors.append("Name must be 100 characters or fewer") + + if errors: + return Err(errors) + return Ok(data) + + +# Caller handles both paths explicitly +match validate_registration(form_data): + case Ok(valid): + user = create_user(valid) + case Err(errs): + return {"errors": errs}, 400 +``` diff --git a/.claude/skills/error-handling/references/typescript-patterns.md b/.claude/skills/error-handling/references/typescript-patterns.md new file mode 100644 index 0000000..cadd186 --- /dev/null +++ b/.claude/skills/error-handling/references/typescript-patterns.md @@ -0,0 +1,451 @@ +# Error Handling — TypeScript Patterns + +## 1. Custom Error Classes + +Define a hierarchy of domain-specific errors so callers can catch at the right granularity. + +```typescript +// error-codes.ts +export const ErrorCode = { + NOT_FOUND: "NOT_FOUND", + VALIDATION_FAILED: "VALIDATION_FAILED", + DUPLICATE_ENTRY: "DUPLICATE_ENTRY", + UNAUTHORIZED: "UNAUTHORIZED", + RATE_LIMITED: "RATE_LIMITED", + EXTERNAL_SERVICE: "EXTERNAL_SERVICE", + INTERNAL: "INTERNAL", +} as const; + +export type ErrorCode = (typeof ErrorCode)[keyof typeof ErrorCode]; + +// app-error.ts +export class AppError extends Error { + public readonly code: ErrorCode; + public readonly details: Record; + + constructor( + message: string, + code: ErrorCode = ErrorCode.INTERNAL, + details: Record = {}, + options?: ErrorOptions + ) { + super(message, options); + this.name = "AppError"; + this.code = code; + this.details = details; + } + + toJSON() { + return { + error: this.code, + message: this.message, + details: this.details, + }; + } +} + +export class NotFoundError extends AppError { + constructor(resource: string, id: string) { + super(`${resource} with id '${id}' not found`, ErrorCode.NOT_FOUND, { + resource, + id, + }); + this.name = "NotFoundError"; + } +} + +export class ValidationError extends AppError { + constructor(field: string, reason: string) { + super( + `Validation failed for '${field}': ${reason}`, + ErrorCode.VALIDATION_FAILED, + { field, reason } + ); + this.name = "ValidationError"; + } +} + +export class ExternalServiceError extends AppError { + constructor(service: string, cause: Error) { + super( + `External service '${service}' failed: ${cause.message}`, + ErrorCode.EXTERNAL_SERVICE, + { service }, + { cause } + ); + this.name = "ExternalServiceError"; + } +} +``` + +## 2. Error Boundaries (React) + +Isolate component failures so a single broken widget does not take down the whole page. + +**Using react-error-boundary (recommended)** + +```typescript +import { + ErrorBoundary, + type FallbackProps, +} from "react-error-boundary"; + +function ErrorFallback({ error, resetErrorBoundary }: FallbackProps) { + return ( +
+

Something went wrong

+ 
{error.message}
+ +
+ ); +} + +function App() { + return ( + { + // Send to error tracking service + reportError({ error, componentStack: info.componentStack }); + }} + onReset={() => { + // Clear any stale state before retry + queryClient.clear(); + }} + > + + + ); +} +``` + +**Granular boundaries per feature** + +```typescript +function DashboardPage() { + return ( +
+ {/* Each widget fails independently */} + + + + + + + + + +
+ ); +} + +function WidgetErrorFallback({ error, resetErrorBoundary }: FallbackProps) { + return ( +
+

This widget failed to load.

+ +
+ ); +} +``` + +**Class-based error boundary (when you need full control)** + +```typescript +import { Component, type ErrorInfo, type ReactNode } from "react"; + +interface Props { + children: ReactNode; + fallback?: ReactNode; +} + +interface State { + hasError: boolean; + error: Error | null; +} + +class ManualErrorBoundary extends Component { + state: State = { hasError: false, error: null }; + + static getDerivedStateFromError(error: Error): State { + return { hasError: true, error }; + } + + componentDidCatch(error: Error, info: ErrorInfo) { + console.error("ErrorBoundary caught:", error, info.componentStack); + } + + render() { + if (this.state.hasError) { + return this.props.fallback ??

Something went wrong.

; + } + return this.props.children; + } +} +``` + +## 3. Retry Pattern + +Retry transient failures with exponential backoff and jitter to avoid thundering herd. + +```typescript +interface RetryOptions { + maxAttempts?: number; + baseDelayMs?: number; + maxDelayMs?: number; + isRetryable?: (error: unknown) => boolean; +} + +async function withRetry( + fn: () => Promise, + options: RetryOptions = {} +): Promise { + const { + maxAttempts = 3, + baseDelayMs = 1000, + maxDelayMs = 30_000, + isRetryable = () => true, + } = options; + + let lastError: unknown; + + for (let attempt = 1; attempt <= maxAttempts; attempt++) { + try { + return await fn(); + } catch (error) { + lastError = error; + + if (attempt === maxAttempts || !isRetryable(error)) { + throw error; + } + + const exponential = baseDelayMs * 2 ** (attempt - 1); + const capped = Math.min(exponential, maxDelayMs); + const jitter = Math.random() * capped; + + console.warn( + `Attempt ${attempt}/${maxAttempts} failed, retrying in ${jitter.toFixed(0)}ms` + ); + await new Promise((resolve) => setTimeout(resolve, jitter)); + } + } + + throw lastError; +} + +// Usage +const data = await withRetry( + () => fetch("/api/config").then((r) => r.json()), + { + maxAttempts: 3, + baseDelayMs: 500, + isRetryable: (err) => + err instanceof TypeError || (err as Response)?.status >= 500, + } +); +``` + +## 4. Graceful Degradation — Feature-Flag Degraded Mode + +When a dependency fails, fall back to a degraded but functional state instead of crashing. + +```typescript +interface FeatureFlags { + enableRecommendations: boolean; + enableRealTimeUpdates: boolean; + enableAdvancedSearch: boolean; +} + +const defaultFlags: FeatureFlags = { + enableRecommendations: true, + enableRealTimeUpdates: true, + enableAdvancedSearch: true, +}; + +async function getFlags(): Promise { + try { + const resp = await fetch("/api/feature-flags"); + if (!resp.ok) throw new Error(`HTTP ${resp.status}`); + return await resp.json(); + } catch { + // Fall back to safe defaults when flag service is unavailable + console.warn("Feature flag service unavailable, using defaults"); + return defaultFlags; + } +} + +// Component that degrades gracefully +function SearchPage() { + const flags = useFeatureFlags(); + + return ( +
+ + {flags.enableAdvancedSearch ? ( + + ) : ( +

+ Advanced search is temporarily unavailable. +

+ )} +
+ ); +} +``` + +## 5. API Error Responses (Express) + +Return consistent, machine-readable error payloads following RFC 7807 Problem Details. + +```typescript +import type { Request, Response, NextFunction } from "express"; + +const STATUS_MAP: Record = { + NOT_FOUND: 404, + VALIDATION_FAILED: 400, + DUPLICATE_ENTRY: 409, + UNAUTHORIZED: 401, + RATE_LIMITED: 429, + EXTERNAL_SERVICE: 502, + INTERNAL: 500, +}; + +function errorHandler( + err: Error, + _req: Request, + res: Response, + _next: NextFunction +) { + if (err instanceof AppError) { + const status = STATUS_MAP[err.code] ?? 500; + res.status(status).json({ + type: `https://docs.example.com/errors/${err.code.toLowerCase()}`, + title: err.code.replace(/_/g, " ").toLowerCase(), + status, + detail: err.message, + ...err.details, + }); + return; + } + + // Unhandled errors - log full details, return generic message + console.error("Unhandled error:", err); + res.status(500).json({ + type: "https://docs.example.com/errors/internal", + title: "Internal Server Error", + status: 500, + detail: "An unexpected error occurred.", + }); +} + +// Register as the last middleware +app.use(errorHandler); +``` + +## 6. Error Logging Integration + +Attach structured context to errors so they are searchable and actionable in observability tools. + +```typescript +interface ErrorContext { + requestId?: string; + userId?: string; + operation?: string; + [key: string]: unknown; +} + +function logError( + message: string, + error: unknown, + context: ErrorContext = {} +): void { + const payload: Record = { + message, + timestamp: new Date().toISOString(), + ...context, + }; + + if (error instanceof AppError) { + payload.errorCode = error.code; + payload.errorMessage = error.message; + payload.errorDetails = error.details; + } else if (error instanceof Error) { + payload.errorType = error.name; + payload.errorMessage = error.message; + payload.stack = error.stack; + } else { + payload.errorRaw = String(error); + } + + // Structured JSON log for ingestion by Datadog, CloudWatch, etc. + console.error(JSON.stringify(payload)); +} + +// Usage +try { + await processOrder(orderId); +} catch (error) { + logError("Order processing failed", error, { + requestId: req.headers["x-request-id"], + operation: "processOrder", + orderId, + }); + throw error; +} +``` + +## 7. Result Pattern + +Use a Result type for operations where failure is an expected outcome. Avoids exception overhead and makes the failure path explicit in the type signature. + +```typescript +type Result = + | { ok: true; value: T } + | { ok: false; error: E }; + +function ok(value: T): Result { + return { ok: true, value }; +} + +function err(error: E): Result { + return { ok: false, error }; +} + +// Usage +function parseAge(input: string): Result { + const age = Number(input); + if (Number.isNaN(age)) return err(`'${input}' is not a number`); + if (age < 0 || age > 150) return err(`Age ${age} out of range (0-150)`); + return ok(age); +} + +function validateRegistration( + data: Record +): Result<{ name: string; age: number }, string[]> { + const errors: string[] = []; + + const ageResult = parseAge(data.age ?? ""); + if (!ageResult.ok) errors.push(ageResult.error); + + const name = data.name?.trim() ?? ""; + if (!name) errors.push("Name is required"); + if (name.length > 100) errors.push("Name must be 100 characters or fewer"); + + if (errors.length > 0) return err(errors); + return ok({ name, age: (ageResult as { ok: true; value: number }).value }); +} + +// Caller +const result = validateRegistration(formData); +if (!result.ok) { + return res.status(400).json({ errors: result.error }); +} +const user = await createUser(result.value); +``` diff --git a/.claude/skills/methodology/executing-plans/SKILL.md b/.claude/skills/executing-plans/SKILL.md similarity index 62% rename from .claude/skills/methodology/executing-plans/SKILL.md rename to .claude/skills/executing-plans/SKILL.md index 92fa729..88105ae 100644 --- a/.claude/skills/methodology/executing-plans/SKILL.md +++ b/.claude/skills/executing-plans/SKILL.md @@ -1,7 +1,7 @@ --- name: executing-plans description: > - Trigger this skill whenever there is a written implementation plan ready to execute, or when the user says "execute", "run the plan", "implement the plan", "start building", or references a plan file. Also activate when using subagent-driven development with independent tasks, when the user wants automated execution with quality gates, or when picking up a previously written plan. If a plan document exists and no one is executing it yet, this is the skill to use. + Use when there is a written implementation plan ready to execute, or when the user says "execute", "run the plan", "implement the plan", "start building", or references a plan file. Also activate when using subagent-driven development with independent tasks, when the user wants automated execution with quality gates, or when picking up a previously written plan. If a plan document exists and no one is executing it yet, this is the skill to use. --- # Executing Plans @@ -110,7 +110,7 @@ After all tasks complete: 2. Review entire implementation against plan 3. Verify all success criteria met 4. Run full test suite -5. Use `finishing-development-branch` skill +5. Use `finishing-a-development-branch` skill ``` --- @@ -172,6 +172,73 @@ RIGHT: Read plan file, extract task details, then implement - Any issues encountered ``` +### Stack-Specific Task Prompt Examples + +**Python/FastAPI:** + +```markdown +## Task: Implement GET /api/users endpoint + +**Context**: FastAPI + SQLAlchemy async + Pydantic v2 +**Files**: src/api/users.py, tests/test_users.py +**Pattern**: Follow src/api/health.py for router setup + +**Steps**: +1. Write test: GET /api/users returns 200 with list +2. Verify test fails (404 — route doesn't exist) +3. Implement: APIRouter, async def, Depends(get_db) +4. Verify test passes +5. Add edge case: GET /api/users/999 returns 404 ProblemDetails + +**Verify**: pytest tests/test_users.py -v (all green) +``` + +**TypeScript/NestJS:** + +```markdown +## Task: Implement UsersController with CRUD + +**Context**: NestJS + Prisma + class-validator DTOs +**Files**: src/users/users.controller.ts, src/users/users.controller.spec.ts +**Pattern**: Follow src/health/ module structure + +**Steps**: +1. Write spec: POST /users returns 201 with user +2. Verify spec fails (404 — no route) +3. Implement: Controller, Service, CreateUserDto with @IsEmail() +4. Verify spec passes +5. Add: GET /users/:id returns 404 for missing user + +**Verify**: npm test -- --testPathPattern=users.controller (all green) +``` + +**React/Next.js:** + +```markdown +## Task: Build UserTable with sorting and pagination + +**Context**: Next.js App Router + TanStack Table + shadcn/ui +**Files**: src/components/user-table.tsx, src/components/user-table.test.tsx +**Pattern**: Follow src/components/data-table.tsx for column defs + +**Steps**: +1. Write test: renders table with user data +2. Verify test fails (component doesn't exist) +3. Implement: columns, DataTable wrapper, sort handlers +4. Verify test passes +5. Add test: clicking column header sorts data + +**Verify**: npx vitest run src/components/user-table.test.tsx (all green) +``` + +### Stack-Specific Verification Commands + +| Stack | Test Command | Full Verify | +|-------|-------------|-------------| +| Python/FastAPI | `pytest tests/test_.py -v` | `pytest -v && ruff check . && mypy src/` | +| TypeScript/NestJS | `npm test -- --testPathPattern=` | `npm test && npm run lint && npm run build` | +| Next.js | `npx vitest run ` | `npm test && next lint && next build` | + ### Code Review Subagent Prompt ```markdown @@ -256,12 +323,12 @@ Before declaring plan execution complete: - [ ] No Critical issues outstanding - [ ] No Important issues outstanding - [ ] Final comprehensive review done -- [ ] Ready for `finishing-development-branch` +- [ ] Ready for `finishing-a-development-branch` --- ## Related Skills -- `methodology/writing-plans` -- Use to create the plan before executing it -- `methodology/dispatching-parallel-agents` -- For coordinating multiple independent agents when plan tasks allow parallelism -- `methodology/verification-before-completion` -- Ensures each task and the final result are properly verified before claiming completion +- `writing-plans` -- Use to create the plan before executing it +- `dispatching-parallel-agents` -- For coordinating multiple independent agents when plan tasks allow parallelism +- `verification-before-completion` -- Ensures each task and the final result are properly verified before claiming completion diff --git a/.claude/skills/methodology/executing-plans/references/execution-checklist.md b/.claude/skills/executing-plans/references/execution-checklist.md similarity index 100% rename from .claude/skills/methodology/executing-plans/references/execution-checklist.md rename to .claude/skills/executing-plans/references/execution-checklist.md diff --git a/.claude/skills/methodology/finishing-development-branch/SKILL.md b/.claude/skills/finishing-a-development-branch/SKILL.md similarity index 72% rename from .claude/skills/methodology/finishing-development-branch/SKILL.md rename to .claude/skills/finishing-a-development-branch/SKILL.md index d5ee1e7..e61ba82 100644 --- a/.claude/skills/methodology/finishing-development-branch/SKILL.md +++ b/.claude/skills/finishing-a-development-branch/SKILL.md @@ -1,7 +1,7 @@ --- -name: finishing-development-branch +name: finishing-a-development-branch description: > - Trigger this skill when implementation is complete and all tests pass, when ready to merge a feature branch, create a PR, or clean up after development. Use whenever you hear "ship it," "ready to merge," "branch is done," or "create a PR." Activate at the end of any feature, bugfix, or chore branch lifecycle to ensure proper verification, option presentation, and worktree cleanup. + Use when implementation is complete and all tests pass, when ready to merge a feature branch, create a PR, or clean up after development. Use whenever you hear "ship it," "ready to merge," "branch is done," or "create a PR." Activate at the end of any feature, bugfix, or chore branch lifecycle to ensure proper verification, option presentation, and worktree cleanup. --- # Finishing a Development Branch @@ -270,6 +270,55 @@ Closes #[issue number] --- +## Stack-Specific Pre-Merge Checklist + +### Python/FastAPI + +```bash +pytest -v --cov=src # Full test suite +ruff check . && ruff format --check . # Lint + format +mypy src/ --strict # Type check +pip-audit # Security audit +alembic upgrade head && alembic check # Verify migrations +``` + +### TypeScript/NestJS + +```bash +npm test # Full test suite +npm run lint # Lint +npm run build # Build (catches type errors) +npm audit --production # Security audit +npx prisma migrate status # Verify migrations +``` + +### Next.js + +```bash +npm test # Tests +next lint # Lint +next build # Build (catches SSR/RSC issues) +``` + +### Stack-Specific PR Description Extras + +**Python/FastAPI PRs** — include: +- Migration included? (alembic revision) +- New dependencies? (requirements.txt changes) +- Async patterns verified? (no blocking calls in async) + +**NestJS PRs** — include: +- New modules registered in AppModule? +- DTOs have class-validator decorators? +- Prisma schema changed? (migration included) + +**Next.js PRs** — include: +- Server vs Client components correct? +- Bundle size impact? +- `'use client'` directives where needed? + +--- + ## Core Principle **"Verify tests → Present options → Execute choice → Clean up"** @@ -284,6 +333,6 @@ Never: ## Related Skills -- `methodology/requesting-code-review` - Use before finishing the branch to get review feedback, especially for Option 2 (Create PR) -- `methodology/verification-before-completion` - Run verification checks before claiming the branch is ready to finish -- `methodology/executing-plans` - If the branch was created from an execution plan, return to the plan to mark tasks complete +- `requesting-code-review` - Use before finishing the branch to get review feedback, especially for Option 2 (Create PR) +- `verification-before-completion` - Run verification checks before claiming the branch is ready to finish +- `executing-plans` - If the branch was created from an execution plan, return to the plan to mark tasks complete diff --git a/.claude/skills/methodology/finishing-development-branch/references/branch-completion-checklist.md b/.claude/skills/finishing-a-development-branch/references/branch-completion-checklist.md similarity index 100% rename from .claude/skills/methodology/finishing-development-branch/references/branch-completion-checklist.md rename to .claude/skills/finishing-a-development-branch/references/branch-completion-checklist.md diff --git a/.claude/skills/frameworks/django/references/django-patterns.md b/.claude/skills/frameworks/django/references/django-patterns.md deleted file mode 100644 index aa72a05..0000000 --- a/.claude/skills/frameworks/django/references/django-patterns.md +++ /dev/null @@ -1,250 +0,0 @@ -# Django Patterns Quick Reference - -## QuerySet Patterns - -### select_related (FK/OneToOne - single JOIN) - -```python -# BAD: N+1 queries -for order in Order.objects.all(): - print(order.customer.name) # Hits DB each iteration - -# GOOD: 1 query with JOIN -for order in Order.objects.select_related("customer"): - print(order.customer.name) - -# Chain through multiple FKs -Order.objects.select_related("customer__company") -``` - -### prefetch_related (M2M/reverse FK - separate query) - -```python -# BAD: N+1 on reverse FK -for author in Author.objects.all(): - print(author.book_set.all()) # Query per author - -# GOOD: 2 queries total -for author in Author.objects.prefetch_related("books"): - print(author.books.all()) - -# Custom prefetch with filtering -from django.db.models import Prefetch - -Author.objects.prefetch_related( - Prefetch("books", queryset=Book.objects.filter(published=True), to_attr="published_books") -) -``` - -### F Objects (reference model fields in queries) - -```python -from django.db.models import F - -Product.objects.filter(stock__lt=F("reorder_level")) # Compare fields -Product.objects.filter(id=1).update(stock=F("stock") - 1) # Atomic update -Order.objects.filter(amount__gt=F("customer__credit_limit")) # Across relations -``` - -### Q Objects (complex lookups with OR/NOT) - -```python -from django.db.models import Q - -# OR -User.objects.filter(Q(role="admin") | Q(is_superuser=True)) - -# NOT -User.objects.filter(~Q(status="banned")) - -# Complex combinations -User.objects.filter( - (Q(role="admin") | Q(role="staff")) & ~Q(status="inactive") -) - -# Dynamic query building -conditions = Q() -if name: conditions &= Q(name__icontains=name) -if email: conditions &= Q(email__icontains=email) -User.objects.filter(conditions) -``` - -### Subquery and OuterRef - -```python -from django.db.models import Subquery, OuterRef, Exists - -# Subquery: latest order date per customer -latest_order = Order.objects.filter( - customer=OuterRef("pk") -).order_by("-created_at").values("created_at")[:1] - -Customer.objects.annotate(last_order=Subquery(latest_order)) - -# Exists: customers with orders -Customer.objects.annotate( - has_orders=Exists(Order.objects.filter(customer=OuterRef("pk"))) -).filter(has_orders=True) -``` - -### Aggregation - -```python -from django.db.models import Count, Sum, Avg - -# Aggregate (returns dict) -Order.objects.aggregate(total=Sum("amount"), avg=Avg("amount")) - -# Annotate (per-row) -Customer.objects.annotate(order_count=Count("orders")) -``` - ---- - -## Model Patterns - -### Abstract Base Model - -```python -class TimestampMixin(models.Model): - created_at = models.DateTimeField(auto_now_add=True) - updated_at = models.DateTimeField(auto_now=True) - - class Meta: - abstract = True # No DB table created - -class Order(TimestampMixin): - amount = models.DecimalField(max_digits=10, decimal_places=2) - # Inherits created_at, updated_at -``` - -### Proxy Model (same table, different behavior) - -```python -class PendingOrderManager(models.Manager): - def get_queryset(self): - return super().get_queryset().filter(status="pending") - -class PendingOrder(Order): - objects = PendingOrderManager() - class Meta: - proxy = True # Same DB table as Order -``` - -### Custom Manager and QuerySet - -```python -class PublishedQuerySet(models.QuerySet): - def published(self): - return self.filter(status="published") - - def by_author(self, author): - return self.filter(author=author) - -class Article(models.Model): - objects = PublishedQuerySet.as_manager() - -# Chainable: Article.objects.published().by_author(user) -``` - ---- - -## View Patterns - -### Class-Based View Mixins - -| Mixin | Purpose | -|-------|---------| -| `LoginRequiredMixin` | Require authentication | -| `PermissionRequiredMixin` | Require specific permission | -| `UserPassesTestMixin` | Custom permission test | -| `FormView` | Handle form display + submission | -| `CreateView` / `UpdateView` | Model form CRUD | -| `ListView` / `DetailView` | Read operations | - -```python -class OrderListView(LoginRequiredMixin, PermissionRequiredMixin, ListView): - model = Order - permission_required = "orders.view_order" - paginate_by = 25 - - def get_queryset(self): - return super().get_queryset().filter(customer=self.request.user) -``` - ---- - -## Django REST Framework Patterns - -### Nested Serializers - -```python -class AddressSerializer(serializers.ModelSerializer): - class Meta: - model = Address - fields = ["street", "city", "zip_code"] - -class CustomerSerializer(serializers.ModelSerializer): - address = AddressSerializer() - - class Meta: - model = Customer - fields = ["id", "name", "address"] - - def create(self, validated_data): - address_data = validated_data.pop("address") - address = Address.objects.create(**address_data) - return Customer.objects.create(address=address, **validated_data) -``` - -### Custom Permissions - -```python -from rest_framework.permissions import BasePermission - -class IsOwner(BasePermission): - def has_object_permission(self, request, view, obj): - return obj.owner == request.user - -class IsAdminOrReadOnly(BasePermission): - def has_permission(self, request, view): - if request.method in ("GET", "HEAD", "OPTIONS"): - return True - return request.user.is_staff - -# Usage -class OrderViewSet(viewsets.ModelViewSet): - permission_classes = [IsAuthenticated, IsOwner] -``` - -### ViewSet Actions - -```python -from rest_framework.decorators import action - -class OrderViewSet(viewsets.ModelViewSet): - queryset = Order.objects.all() - serializer_class = OrderSerializer - - @action(detail=True, methods=["post"]) - def cancel(self, request, pk=None): - order = self.get_object() - order.cancel() - return Response({"status": "cancelled"}) - - @action(detail=False, methods=["get"]) - def summary(self, request): - return Response(self.get_queryset().aggregate(total=Sum("amount"), count=Count("id"))) -``` - -### Filtering (django-filter) - -```python -class OrderFilter(django_filters.FilterSet): - min_amount = django_filters.NumberFilter(field_name="amount", lookup_expr="gte") - max_amount = django_filters.NumberFilter(field_name="amount", lookup_expr="lte") - - class Meta: - model = Order - fields = ["status", "customer"] -``` diff --git a/.claude/skills/frameworks/fastapi/references/fastapi-project-structure.md b/.claude/skills/frameworks/fastapi/references/fastapi-project-structure.md deleted file mode 100644 index 923de12..0000000 --- a/.claude/skills/frameworks/fastapi/references/fastapi-project-structure.md +++ /dev/null @@ -1,229 +0,0 @@ -# FastAPI Project Structure Reference - -## Small Project (1-5 endpoints, single module) - -``` -project/ -├── main.py # App factory, routes, startup -├── models.py # Pydantic schemas + SQLAlchemy models -├── database.py # DB connection, session factory -├── config.py # Settings via pydantic-settings -├── requirements.txt -├── .env -└── tests/ - ├── conftest.py # Fixtures (test client, test DB) - └── test_main.py -``` - -**When to use**: Prototypes, microservices, internal tools, single-domain APIs. - -**`main.py` structure**: -```python -from fastapi import FastAPI -from contextlib import asynccontextmanager - -@asynccontextmanager -async def lifespan(app: FastAPI): - # startup - yield - # shutdown - -app = FastAPI(lifespan=lifespan) - -@app.get("/health") -async def health(): return {"status": "ok"} -``` - ---- - -## Medium Project (5-20 endpoints, feature-grouped) - -``` -project/ -├── app/ -│ ├── __init__.py -│ ├── main.py # App factory, include routers -│ ├── config.py # Settings (pydantic-settings) -│ ├── database.py # Engine, SessionLocal, Base -│ ├── dependencies.py # Shared deps (get_db, get_current_user) -│ ├── exceptions.py # Custom exception handlers -│ ├── middleware.py # CORS, logging, timing middleware -│ │ -│ ├── auth/ -│ │ ├── __init__.py -│ │ ├── router.py # POST /login, POST /register -│ │ ├── schemas.py # LoginRequest, TokenResponse -│ │ ├── models.py # User SQLAlchemy model -│ │ ├── service.py # Business logic (hash, verify, tokens) -│ │ └── dependencies.py # get_current_user, require_role -│ │ -│ ├── items/ -│ │ ├── __init__.py -│ │ ├── router.py # CRUD endpoints -│ │ ├── schemas.py # ItemCreate, ItemRead, ItemUpdate -│ │ ├── models.py # Item SQLAlchemy model -│ │ └── service.py # Business logic -│ │ -│ └── shared/ -│ ├── __init__.py -│ ├── pagination.py # Pagination params + response schema -│ └── filters.py # Common query filter patterns -│ -├── alembic/ # DB migrations -│ ├── env.py -│ └── versions/ -├── alembic.ini -├── requirements.txt -├── pyproject.toml -├── Dockerfile -├── docker-compose.yml -└── tests/ - ├── conftest.py - ├── auth/ - │ └── test_router.py - └── items/ - ├── test_router.py - └── test_service.py -``` - -**When to use**: Multi-feature APIs, team projects, typical SaaS backends. - -**Key patterns**: -- Each feature gets its own directory with router, schemas, models, service -- `router.py` uses `APIRouter(prefix="/items", tags=["items"])` -- `main.py` includes routers: `app.include_router(items.router)` -- Shared deps in root `dependencies.py`, feature-specific in feature dir - ---- - -## Large Project (20+ endpoints, domain-driven) - -``` -project/ -├── src/ -│ ├── __init__.py -│ ├── main.py # App factory only -│ ├── config.py # Layered settings -│ │ -│ ├── core/ # Framework-level concerns -│ │ ├── __init__.py -│ │ ├── database.py # Engine, session management -│ │ ├── security.py # JWT, hashing, RBAC -│ │ ├── exceptions.py # Base exceptions + handlers -│ │ ├── middleware.py # All middleware stack -│ │ ├── dependencies.py # Cross-cutting deps -│ │ ├── events.py # Domain event bus -│ │ └── pagination.py # Cursor + offset pagination -│ │ -│ ├── domain/ # Business logic (framework-agnostic) -│ │ ├── users/ -│ │ │ ├── __init__.py -│ │ │ ├── entity.py # Domain entity (plain dataclass) -│ │ │ ├── repository.py # Abstract repository interface -│ │ │ ├── service.py # Business rules -│ │ │ └── events.py # Domain events -│ │ ├── orders/ -│ │ │ └── ... -│ │ └── payments/ -│ │ └── ... -│ │ -│ ├── infrastructure/ # External system adapters -│ │ ├── database/ -│ │ │ ├── models.py # All SQLAlchemy models -│ │ │ ├── repositories/ # Concrete repo implementations -│ │ │ │ ├── user_repo.py -│ │ │ │ └── order_repo.py -│ │ │ └── migrations/ # Alembic -│ │ ├── cache/ -│ │ │ └── redis_client.py -│ │ ├── email/ -│ │ │ └── smtp_service.py -│ │ └── external/ -│ │ └── stripe_client.py -│ │ -│ └── api/ # HTTP layer only -│ ├── __init__.py -│ ├── v1/ -│ │ ├── __init__.py # v1 router aggregator -│ │ ├── users.py # Thin: parse request -> call service -> format response -│ │ ├── orders.py -│ │ └── payments.py -│ ├── v2/ -│ │ └── ... -│ ├── schemas/ # Request/response schemas -│ │ ├── user_schemas.py -│ │ ├── order_schemas.py -│ │ └── common.py -│ ├── dependencies.py # API-layer deps -│ └── websockets/ -│ └── notifications.py -│ -├── tests/ -│ ├── conftest.py -│ ├── unit/ -│ │ ├── domain/ -│ │ │ └── test_user_service.py -│ │ └── ... -│ ├── integration/ -│ │ ├── test_user_api.py -│ │ └── test_order_flow.py -│ └── e2e/ -│ └── test_checkout.py -│ -├── scripts/ # Dev/ops scripts -│ ├── seed_db.py -│ └── migrate.py -├── pyproject.toml -├── Dockerfile -├── docker-compose.yml -└── Makefile -``` - -**When to use**: Complex domains, multiple teams, long-lived products. - -**Key patterns**: -- **Domain layer** has zero framework imports (testable in isolation) -- **Infrastructure** implements domain interfaces (repository pattern) -- **API layer** is thin: validation, auth, call service, return schema -- API versioning via `/api/v1/`, `/api/v2/` -- Separate unit, integration, and e2e test directories - ---- - -## File Responsibilities - -| File | Responsibility | Dependencies | -|------|---------------|-------------| -| `router.py` | HTTP handling, request parsing, response formatting | schemas, service, dependencies | -| `schemas.py` | Pydantic models for request/response validation | None (or shared schemas) | -| `models.py` | SQLAlchemy/ODM models (DB table mapping) | database | -| `service.py` | Business logic, orchestration | repository/models, external services | -| `dependencies.py` | FastAPI `Depends()` callables | database, config, auth | -| `exceptions.py` | Custom exceptions + handlers | None | -| `config.py` | `BaseSettings` with env loading | None | - -## Router Registration Pattern - -```python -# app/main.py -from fastapi import FastAPI -from app.auth.router import router as auth_router -from app.items.router import router as items_router - -def create_app() -> FastAPI: - app = FastAPI(title="My API") - app.include_router(auth_router) - app.include_router(items_router) - return app - -app = create_app() -``` - -```python -# app/items/router.py -from fastapi import APIRouter, Depends -router = APIRouter(prefix="/items", tags=["items"]) - -@router.get("/") -async def list_items(db=Depends(get_db)): ... -``` diff --git a/.claude/skills/frameworks/nextjs/references/nextjs-patterns.md b/.claude/skills/frameworks/nextjs/references/nextjs-patterns.md deleted file mode 100644 index a830746..0000000 --- a/.claude/skills/frameworks/nextjs/references/nextjs-patterns.md +++ /dev/null @@ -1,237 +0,0 @@ -# Next.js Patterns Quick Reference - -> App Router (Next.js 13.4+). Pages Router patterns not covered. - -## App Router File Conventions - -| File | Purpose | Renders | -|------|---------|---------| -| `page.tsx` | Route UI (makes route publicly accessible) | Server Component | -| `layout.tsx` | Shared UI wrapping children (preserved on nav) | Server Component | -| `template.tsx` | Like layout but re-mounts on navigation | Server Component | -| `loading.tsx` | Instant loading UI (Suspense boundary) | Server Component | -| `error.tsx` | Error boundary for segment | **Client Component** | -| `not-found.tsx` | UI for `notFound()` calls | Server Component | -| `route.ts` | API endpoint (GET, POST, etc.) | N/A | -| `default.tsx` | Fallback for parallel routes | Server Component | -| `middleware.ts` | Runs before requests (root only) | Edge Runtime | -| `opengraph-image.tsx` | Dynamic OG image generation | Edge Runtime | - -### Route Segment Folders - -| Pattern | Example | Purpose | -|---------|---------|---------| -| Static | `app/about/page.tsx` | `/about` | -| Dynamic | `app/blog/[slug]/page.tsx` | `/blog/hello-world` | -| Catch-all | `app/docs/[...slug]/page.tsx` | `/docs/a/b/c` | -| Optional catch-all | `app/docs/[[...slug]]/page.tsx` | `/docs` or `/docs/a/b` | -| Route group | `app/(marketing)/about/page.tsx` | Groups without URL segment | -| Parallel route | `app/@modal/login/page.tsx` | Simultaneous route slots | -| Intercepted route | `app/(.)photo/[id]/page.tsx` | Intercept navigation | - ---- - -## Caching Layers Summary - -| Layer | What | Where | Duration | Opt-out | -|-------|------|-------|----------|---------| -| Request Memoization | `fetch()` dedup in single render | Server | Per request | `AbortController` | -| Data Cache | `fetch()` results | Server | Persistent | `cache: 'no-store'` or `revalidate: 0` | -| Full Route Cache | Static HTML + RSC payload | Server | Persistent | Dynamic functions or `revalidate` | -| Router Cache | RSC payload | Client | Session (30s dynamic, 5min static) | `router.refresh()` | - -### Revalidation Strategies - -```typescript -// Time-based -fetch(url, { next: { revalidate: 60 } }); // Revalidate every 60s - -// On-demand (from API route or Server Action) -import { revalidatePath, revalidateTag } from "next/cache"; -revalidatePath("/blog"); // Revalidate path -revalidateTag("posts"); // Revalidate by tag - -// Tag a fetch for on-demand revalidation -fetch(url, { next: { tags: ["posts"] } }); - -// Opt out entirely -fetch(url, { cache: "no-store" }); - -// Route segment config -export const dynamic = "force-dynamic"; // Always dynamic -export const revalidate = 60; // Segment-level ISR -``` - ---- - -## Data Fetching Patterns - -### Server Component (default, preferred) - -```typescript -// Async Server Component - fetch directly -async function BlogPage() { - const posts = await fetch("https://api.example.com/posts", { - next: { revalidate: 3600 } - }).then(r => r.json()); - - return ; -} -``` - -### Parallel Data Fetching - -```typescript -async function Dashboard() { - // Start all fetches simultaneously - const [user, orders, stats] = await Promise.all([ - getUser(), - getOrders(), - getStats(), - ]); - return ; -} -``` - -### Streaming with Suspense - -```typescript -export default function Page() { - return ( -
-

Dashboard

- {/* Shows instantly */} - - - {/* Streams in when ready */} - }> - - - - }> - - -
- ); -} -``` - -### Cached Server Functions - -```typescript -import { cache } from "react"; - -// Deduplicated across components in same request -export const getUser = cache(async (id: string) => { - const res = await fetch(`/api/users/${id}`); - return res.json(); -}); -``` - ---- - -## Server Action Patterns - -### Basic Form Action - -```typescript -// app/actions.ts -"use server"; - -import { revalidatePath } from "next/cache"; -import { redirect } from "next/navigation"; - -export async function createPost(formData: FormData) { - const title = formData.get("title") as string; - const body = formData.get("body") as string; - - await db.post.create({ data: { title, body } }); - revalidatePath("/posts"); - redirect("/posts"); -} -``` - -```typescript -// In a Server Component -import { createPost } from "./actions"; - -export default function NewPost() { - return ( -
- - - - -
- - -
-
-``` - ---- - -## Modal Overlay - -```html - -
- -
-
-

Confirm Action

- -
-

- Are you sure you want to proceed? This action cannot be undone. -

-
- - -
-
-
-``` - ---- - -## Sidebar Layout - -```html -
- - - - -
-
-

Dashboard

-
- -
-
-
-
-``` - ---- - -## Tips - -- Use `transition` and `hover:` for interactive feedback -- Use `focus-visible:` instead of `focus:` for keyboard-only focus rings -- Use `dark:` variants when supporting dark mode -- Prefer `gap-*` over margin utilities for flex/grid spacing -- Use `max-w-7xl mx-auto px-4 sm:px-6 lg:px-8` as a standard container diff --git a/.claude/skills/languages/SKILL.md b/.claude/skills/languages/SKILL.md new file mode 100644 index 0000000..58d56d8 --- /dev/null +++ b/.claude/skills/languages/SKILL.md @@ -0,0 +1,62 @@ +--- +name: languages +description: > + Use when working with Python, TypeScript, or JavaScript language-specific patterns — including type hints, generics, async/await, dataclasses, Pydantic, PEP 8, strict mode, tsconfig.json, Zod schemas, ESM/CJS, destructuring, optional chaining, or language idioms. +--- + +# Languages + +## When to Use + +- Python files (.py) — type hints, async/await, dataclasses, Pydantic, context managers, PEP 8 +- TypeScript files (.ts, .tsx) — strict mode, generics, utility types, Zod, discriminated unions +- JavaScript files (.js, .mjs, .cjs) — ES6+ patterns, ESM/CJS, ESLint, modern syntax +- Language-specific idioms, package management (pip, pnpm), or migration between languages + +## When NOT to Use + +- Framework-specific patterns — use `backend-frameworks` or `frontend` +- Testing — use `testing` +- Database queries — use `databases` + +--- + +## Quick Reference + +| Language | Reference | Key features | +|----------|-----------|-------------| +| Python | `references/python.md` | Type hints, dataclasses, Pydantic, asyncio, context managers, PEP 8 | +| TypeScript | `references/typescript.md` | Strict mode, generics, utility types, Zod, discriminated unions, satisfies | +| JavaScript | `references/javascript.md` | ES6+, async/await, ESM/CJS, destructuring, private fields, structuredClone | + +--- + +## Best Practices + +1. **Use type hints on all public functions** (Python) / **enable strict mode** (TypeScript). +2. **Prefer dataclasses or Pydantic for structured data** (Python) / **interfaces for object shapes** (TypeScript). +3. **Use context managers for resource management** (Python). +4. **Never use `any`** — use `unknown` instead (TypeScript). +5. **Use `const` by default, `let` when needed, never `var`** (JavaScript). +6. **Validate external data at boundaries** — Zod (TypeScript) or Pydantic (Python). +7. **Handle all promise rejections** (JavaScript/TypeScript). +8. **Follow PEP 8 / ESLint + Prettier** consistently. + +## Common Pitfalls + +1. **Mutable default arguments** (Python) — use `None` with default in function body. +2. **Blocking calls inside async functions** (Python) — use `asyncio`-compatible libraries. +3. **Overusing type assertions `as`** (TypeScript) — use type guards instead. +4. **Implicit type coercion** (JavaScript) — always use `===` and `!==`. +5. **Forgetting `await`** (all three languages). +6. **Circular imports** (Python) / **circular dependencies** (TypeScript). +7. **`this` binding in callbacks** (JavaScript) — use arrow functions. +8. **Using enums instead of const objects** (TypeScript). + +--- + +## Related Skills + +- `backend-frameworks` — Framework-specific patterns +- `testing` — Language-specific test frameworks +- `error-handling` — Exception handling patterns diff --git a/.claude/skills/languages/javascript/references/modern-js-patterns.md b/.claude/skills/languages/javascript/references/modern-js-patterns.md deleted file mode 100644 index f4599bb..0000000 --- a/.claude/skills/languages/javascript/references/modern-js-patterns.md +++ /dev/null @@ -1,247 +0,0 @@ -# Modern JavaScript Patterns Quick Reference - -> ES2020+ patterns. All examples work in current Node.js (18+) and modern browsers. - -## Destructuring Tricks - -```javascript -// Nested destructuring with rename and default -const { data: { users: members = [] } = {} } = response; - -// Array destructuring: skip elements -const [first, , third] = [1, 2, 3]; - -// Swap variables -[a, b] = [b, a]; - -// Rest in both arrays and objects -const { id, ...rest } = user; -const [head, ...tail] = items; - -// Destructure function parameters -function draw({ x = 0, y = 0, color = "black" } = {}) { /* ... */ } - -// Dynamic property destructuring -const key = "name"; -const { [key]: value } = { name: "Alice" }; // value = "Alice" - -// Destructure from iterables -const [a, b] = new Map([["a", 1], ["b", 2]]); -``` - -## Optional Chaining (?.) - -```javascript -// Property access -const city = user?.address?.city; - -// Method call (only calls if method exists) -const result = api?.getData?.(); - -// Bracket notation -const val = obj?.["dynamic-key"]; - -// Array index -const first = arr?.[0]; - -// Combine with nullish coalescing -const name = user?.profile?.name ?? "Anonymous"; - -// Short-circuit: stops evaluating after first nullish -const len = response?.data?.items?.length; // undefined if any is nullish -``` - -## Nullish Coalescing (??) vs OR (||) - -```javascript -// ?? only triggers on null/undefined (NOT 0, "", false) -0 ?? "fallback" // 0 -"" ?? "fallback" // "" -null ?? "fallback" // "fallback" - -// || triggers on any falsy value -0 || "fallback" // "fallback" -"" || "fallback" // "fallback" -null || "fallback" // "fallback" - -// Use ?? for values where 0/empty string are valid -const port = config.port ?? 3000; -const title = config.title ?? "Untitled"; -``` - -## Logical Assignment Operators - -```javascript -// ??= assigns only if current value is null/undefined -user.name ??= "Anonymous"; -// Equivalent: user.name = user.name ?? "Anonymous" - -// ||= assigns if current value is falsy -opts.verbose ||= false; -// Equivalent: opts.verbose = opts.verbose || false - -// &&= assigns only if current value is truthy -user.token &&= encrypt(user.token); -// Equivalent: user.token = user.token && encrypt(user.token) -``` - -## structuredClone (Deep Copy) - -```javascript -// Deep clone objects, arrays, Maps, Sets, Dates, RegExp, etc. -const original = { date: new Date(), nested: { arr: [1, 2] } }; -const clone = structuredClone(original); -clone.nested.arr.push(3); // original not affected - -// Works with circular references -const obj = { self: null }; -obj.self = obj; -const copy = structuredClone(obj); // OK - -// Does NOT clone: functions, DOM nodes, symbols, prototype chain -// Throws on: functions, Error objects (in some engines) -``` - -## Proxy - -```javascript -// Validation proxy -const validated = new Proxy({}, { - set(target, prop, value) { - if (prop === "age" && (typeof value !== "number" || value < 0)) { - throw new TypeError("Age must be a non-negative number"); - } - target[prop] = value; - return true; - } -}); - -// Read-only proxy -function readonly(target) { - return new Proxy(target, { - set() { throw new Error("Read-only object"); }, - deleteProperty() { throw new Error("Read-only object"); } - }); -} - -// Default values proxy -function withDefaults(target, defaults) { - return new Proxy(target, { - get(obj, prop) { - return prop in obj ? obj[prop] : defaults[prop]; - } - }); -} -const config = withDefaults({}, { theme: "dark", lang: "en" }); -config.theme; // "dark" - -// Logging / observation proxy -function observable(target, onChange) { - return new Proxy(target, { - set(obj, prop, value) { - const old = obj[prop]; - obj[prop] = value; - onChange(prop, old, value); - return true; - } - }); -} -``` - -## Generators - -```javascript -// Basic generator -function* range(start, end, step = 1) { - for (let i = start; i < end; i += step) { - yield i; - } -} -for (const n of range(0, 10, 2)) { /* 0, 2, 4, 6, 8 */ } - -// Infinite sequence -function* ids() { - let id = 0; - while (true) yield id++; -} -const gen = ids(); -gen.next().value; // 0 -gen.next().value; // 1 - -// Delegate to another generator -function* concat(...iterables) { - for (const it of iterables) { - yield* it; - } -} - -// Two-way communication -function* stateMachine() { - let input; - while (true) { - input = yield `received: ${input}`; - } -} -const sm = stateMachine(); -sm.next(); // { value: "received: undefined" } -sm.next("hello"); // { value: "received: hello" } -``` - -## Async Iterators - -```javascript -// for-await-of -async function processStream(stream) { - for await (const chunk of stream) { - console.log(chunk); - } -} - -// Async generator -async function* fetchPages(url) { - let page = 1; - while (true) { - const res = await fetch(`${url}?page=${page}`); - const data = await res.json(); - if (data.items.length === 0) return; - yield data.items; - page++; - } -} - -for await (const items of fetchPages("/api/users")) { - console.log(items); -} - -``` - -## Other Modern Patterns - -```javascript -// Object.groupBy (ES2024) -const grouped = Object.groupBy(users, u => u.role); - -// at() - negative indexing -[1, 2, 3].at(-1); // 3 - -// Object.hasOwn (replaces hasOwnProperty) -Object.hasOwn(obj, "key"); // true/false - -// Error cause chaining -throw new Error("DB failed", { cause: originalError }); - -// AbortSignal.timeout (built-in timeout) -fetch(url, { signal: AbortSignal.timeout(5000) }); - -// using keyword (explicit resource management, ES2024+) -{ using handle = openFile("data.txt"); } // auto-disposed at block exit -``` - -## Promise Combinators - -| Method | Settles when | Returns | -|--------|-------------|---------| -| `Promise.all(ps)` | All fulfill or one rejects | Array of values | -| `Promise.allSettled(ps)` | All settle | Array of `{status, value/reason}` | -| `Promise.race(ps)` | First settles | First value or rejection | -| `Promise.any(ps)` | First fulfills | First value (AggregateError if all reject) | diff --git a/.claude/skills/languages/python/references/type-hints-reference.md b/.claude/skills/languages/python/references/type-hints-reference.md deleted file mode 100644 index 431a160..0000000 --- a/.claude/skills/languages/python/references/type-hints-reference.md +++ /dev/null @@ -1,216 +0,0 @@ -# Python Type Hints Quick Reference - -> Python 3.10+ syntax preferred. For 3.9, use `from __future__ import annotations`. - -## Basic Types - -| Type | Example | Notes | -|------|---------|-------| -| `int` | `x: int = 1` | | -| `float` | `x: float = 1.0` | | -| `str` | `x: str = "hi"` | | -| `bool` | `x: bool = True` | | -| `bytes` | `x: bytes = b"hi"` | | -| `None` | `x: None = None` | Use as return type for side-effect functions | -| `object` | `x: object` | Accepts anything, but no attribute access | -| `Any` | `x: Any` | Escapes type checking entirely | - -## Collection Types (3.10+) - -| Type | Example | Notes | -|------|---------|-------| -| `list[int]` | `x: list[int] = [1, 2]` | Mutable sequence | -| `tuple[int, str]` | `x: tuple[int, str]` | Fixed length | -| `tuple[int, ...]` | `x: tuple[int, ...]` | Variable length | -| `dict[str, int]` | `x: dict[str, int]` | | -| `set[str]` | `x: set[str]` | | -| `frozenset[str]` | `x: frozenset[str]` | | - -## Union and Optional - -```python -# 3.10+ syntax -def f(x: int | str) -> None: ... -def g(x: int | None = None) -> None: ... - -# Pre-3.10 -from typing import Union, Optional -def f(x: Union[int, str]) -> None: ... -def g(x: Optional[int] = None) -> None: ... -``` - -## TypeAlias - -```python -from typing import TypeAlias - -# Explicit alias (3.10+) -Vector: TypeAlias = list[float] - -# 3.12+ syntax -type Vector = list[float] -type Tree[T] = T | list["Tree[T]"] # recursive -``` - -## Generics with TypeVar - -```python -from typing import TypeVar - -T = TypeVar("T") -K = TypeVar("K", bound=str) # upper bound -N = TypeVar("N", int, float) # constrained - -def first(items: list[T]) -> T: - return items[0] - -# 3.12+ syntax (no TypeVar needed) -def first[T](items: list[T]) -> T: - return items[0] -``` - -## ParamSpec and Concatenate - -```python -from typing import ParamSpec, Concatenate, Callable - -P = ParamSpec("P") -T = TypeVar("T") - -# Preserve function signatures through decorators -def logged(fn: Callable[P, T]) -> Callable[P, T]: - def wrapper(*args: P.args, **kwargs: P.kwargs) -> T: - print(f"Calling {fn.__name__}") - return fn(*args, **kwargs) - return wrapper - -# Add a parameter to a function signature -def with_user( - fn: Callable[Concatenate[User, P], T] -) -> Callable[P, T]: - def wrapper(*args: P.args, **kwargs: P.kwargs) -> T: - return fn(get_current_user(), *args, **kwargs) - return wrapper -``` - -## Protocol (Structural Typing) - -```python -from typing import Protocol, runtime_checkable - -class Renderable(Protocol): - def render(self) -> str: ... - -class Widget(Protocol): - name: str - def resize(self, width: int, height: int) -> None: ... - -# Any class with matching methods satisfies the protocol -class Button: - def render(self) -> str: - return " - - ); -} - -function App() { - return ( - { - // Send to error tracking service - reportError({ error, componentStack: info.componentStack }); - }} - onReset={() => { - // Clear any stale state before retry - queryClient.clear(); - }} - > - - - ); -} -``` - -**Granular boundaries per feature** - -```typescript -function DashboardPage() { - return ( -
- {/* Each widget fails independently */} - - - - - - - - - -
- ); -} - -function WidgetErrorFallback({ error, resetErrorBoundary }: FallbackProps) { - return ( -
-

This widget failed to load.

- -
- ); -} -``` - -**Class-based error boundary (when you need full control)** - -```typescript -import { Component, type ErrorInfo, type ReactNode } from "react"; - -interface Props { - children: ReactNode; - fallback?: ReactNode; -} - -interface State { - hasError: boolean; - error: Error | null; -} - -class ManualErrorBoundary extends Component { - state: State = { hasError: false, error: null }; - - static getDerivedStateFromError(error: Error): State { - return { hasError: true, error }; - } - - componentDidCatch(error: Error, info: ErrorInfo) { - console.error("ErrorBoundary caught:", error, info.componentStack); - } - - render() { - if (this.state.hasError) { - return this.props.fallback ??

Something went wrong.

; - } - return this.props.children; - } -} -``` - ---- - -### 3. Retry Patterns - -Retry transient failures with exponential backoff and jitter to avoid thundering herd. - -**Python - Retry decorator with exponential backoff** - -```python -import asyncio -import random -import logging -from functools import wraps -from typing import TypeVar, Callable, Awaitable - -logger = logging.getLogger(__name__) - -T = TypeVar("T") - - -def retry( - max_attempts: int = 3, - base_delay: float = 1.0, - max_delay: float = 30.0, - retryable: tuple[type[Exception], ...] = (Exception,), -) -> Callable: - """Retry decorator with exponential backoff and full jitter. - - Args: - max_attempts: Total number of attempts including the first call. - base_delay: Initial delay in seconds before the first retry. - max_delay: Upper bound on the computed delay. - retryable: Exception types eligible for retry. - """ - - def decorator(fn: Callable[..., Awaitable[T]]) -> Callable[..., Awaitable[T]]: - @wraps(fn) - async def wrapper(*args, **kwargs) -> T: - last_exc: Exception | None = None - for attempt in range(1, max_attempts + 1): - try: - return await fn(*args, **kwargs) - except retryable as exc: - last_exc = exc - if attempt == max_attempts: - break - delay = min(base_delay * (2 ** (attempt - 1)), max_delay) - jitter = random.uniform(0, delay) - logger.warning( - "Attempt %d/%d failed (%s), retrying in %.2fs", - attempt, - max_attempts, - exc, - jitter, - ) - await asyncio.sleep(jitter) - raise last_exc # type: ignore[misc] - - return wrapper - - return decorator - - -# Usage -@retry(max_attempts=3, base_delay=0.5, retryable=(ConnectionError, TimeoutError)) -async def fetch_remote_config(url: str) -> dict: - async with httpx.AsyncClient(timeout=5) as client: - resp = await client.get(url) - resp.raise_for_status() - return resp.json() -``` - -**TypeScript - Retry wrapper** - -```typescript -interface RetryOptions { - maxAttempts?: number; - baseDelayMs?: number; - maxDelayMs?: number; - isRetryable?: (error: unknown) => boolean; -} - -async function withRetry( - fn: () => Promise, - options: RetryOptions = {} -): Promise { - const { - maxAttempts = 3, - baseDelayMs = 1000, - maxDelayMs = 30_000, - isRetryable = () => true, - } = options; - - let lastError: unknown; - - for (let attempt = 1; attempt <= maxAttempts; attempt++) { - try { - return await fn(); - } catch (error) { - lastError = error; - - if (attempt === maxAttempts || !isRetryable(error)) { - throw error; - } - - const exponential = baseDelayMs * 2 ** (attempt - 1); - const capped = Math.min(exponential, maxDelayMs); - const jitter = Math.random() * capped; - - console.warn( - `Attempt ${attempt}/${maxAttempts} failed, retrying in ${jitter.toFixed(0)}ms` - ); - await new Promise((resolve) => setTimeout(resolve, jitter)); - } - } - - throw lastError; -} - -// Usage -const data = await withRetry( - () => fetch("/api/config").then((r) => r.json()), - { - maxAttempts: 3, - baseDelayMs: 500, - isRetryable: (err) => - err instanceof TypeError || (err as Response)?.status >= 500, - } -); -``` - ---- - -### 4. Graceful Degradation - -When a dependency fails, fall back to a degraded but functional state instead of crashing. - -**Circuit breaker pattern (Python)** - -```python -import time -from enum import Enum - - -class CircuitState(Enum): - CLOSED = "closed" # normal operation - OPEN = "open" # failing, reject immediately - HALF_OPEN = "half_open" # testing recovery - - -class CircuitBreaker: - """Prevents cascading failures by short-circuiting calls to an unhealthy dependency.""" - - def __init__( - self, - failure_threshold: int = 5, - recovery_timeout: float = 30.0, - ) -> None: - self.failure_threshold = failure_threshold - self.recovery_timeout = recovery_timeout - self.state = CircuitState.CLOSED - self.failure_count = 0 - self.last_failure_time = 0.0 - - def _trip(self) -> None: - self.state = CircuitState.OPEN - self.last_failure_time = time.monotonic() - - def _reset(self) -> None: - self.state = CircuitState.CLOSED - self.failure_count = 0 - - async def call(self, fn, *args, fallback=None, **kwargs): - if self.state == CircuitState.OPEN: - if time.monotonic() - self.last_failure_time > self.recovery_timeout: - self.state = CircuitState.HALF_OPEN - else: - if fallback is not None: - return fallback() if callable(fallback) else fallback - raise ExternalServiceError("circuit-breaker", RuntimeError("Circuit open")) - - try: - result = await fn(*args, **kwargs) - if self.state == CircuitState.HALF_OPEN: - self._reset() - return result - except Exception as exc: - self.failure_count += 1 - if self.failure_count >= self.failure_threshold: - self._trip() - raise - - -# Usage -recommendations_circuit = CircuitBreaker(failure_threshold=3, recovery_timeout=60) - -async def get_recommendations(user_id: str) -> list[dict]: - return await recommendations_circuit.call( - recommendation_service.fetch, - user_id, - fallback=lambda: [], # empty list when service is down - ) -``` - -**Feature-flag degraded mode (TypeScript)** - -```typescript -interface FeatureFlags { - enableRecommendations: boolean; - enableRealTimeUpdates: boolean; - enableAdvancedSearch: boolean; -} - -const defaultFlags: FeatureFlags = { - enableRecommendations: true, - enableRealTimeUpdates: true, - enableAdvancedSearch: true, -}; - -async function getFlags(): Promise { - try { - const resp = await fetch("/api/feature-flags"); - if (!resp.ok) throw new Error(`HTTP ${resp.status}`); - return await resp.json(); - } catch { - // Fall back to safe defaults when flag service is unavailable - console.warn("Feature flag service unavailable, using defaults"); - return defaultFlags; - } -} - -// Component that degrades gracefully -function SearchPage() { - const flags = useFeatureFlags(); - - return ( -
- - {flags.enableAdvancedSearch ? ( - - ) : ( -

- Advanced search is temporarily unavailable. -

- )} -
- ); -} -``` - ---- - -### 5. API Error Responses - -Return consistent, machine-readable error payloads following RFC 7807 Problem Details. - -**Python (FastAPI)** - -```python -from fastapi import FastAPI, Request -from fastapi.responses import JSONResponse -from starlette.status import ( - HTTP_400_BAD_REQUEST, - HTTP_404_NOT_FOUND, - HTTP_429_TOO_MANY_REQUESTS, - HTTP_500_INTERNAL_SERVER_ERROR, -) - -app = FastAPI() - -# Map domain error codes to HTTP status codes -STATUS_MAP: dict[ErrorCode, int] = { - ErrorCode.NOT_FOUND: HTTP_404_NOT_FOUND, - ErrorCode.VALIDATION_FAILED: HTTP_400_BAD_REQUEST, - ErrorCode.DUPLICATE_ENTRY: 409, - ErrorCode.UNAUTHORIZED: 401, - ErrorCode.RATE_LIMITED: HTTP_429_TOO_MANY_REQUESTS, - ErrorCode.EXTERNAL_SERVICE: 502, - ErrorCode.INTERNAL: HTTP_500_INTERNAL_SERVER_ERROR, -} - - -@app.exception_handler(AppError) -async def app_error_handler(request: Request, exc: AppError) -> JSONResponse: - status = STATUS_MAP.get(exc.code, 500) - return JSONResponse( - status_code=status, - content={ - "type": f"https://docs.example.com/errors/{exc.code.value.lower()}", - "title": exc.code.value.replace("_", " ").title(), - "status": status, - "detail": str(exc), - **exc.details, - }, - ) - - -@app.exception_handler(Exception) -async def unhandled_error_handler(request: Request, exc: Exception) -> JSONResponse: - # Never leak internal details to the client - logger.exception("Unhandled exception on %s %s", request.method, request.url.path) - return JSONResponse( - status_code=500, - content={ - "type": "https://docs.example.com/errors/internal", - "title": "Internal Server Error", - "status": 500, - "detail": "An unexpected error occurred.", - }, - ) -``` - -**TypeScript (Express middleware)** - -```typescript -import type { Request, Response, NextFunction } from "express"; - -const STATUS_MAP: Record = { - NOT_FOUND: 404, - VALIDATION_FAILED: 400, - DUPLICATE_ENTRY: 409, - UNAUTHORIZED: 401, - RATE_LIMITED: 429, - EXTERNAL_SERVICE: 502, - INTERNAL: 500, -}; - -function errorHandler( - err: Error, - _req: Request, - res: Response, - _next: NextFunction -) { - if (err instanceof AppError) { - const status = STATUS_MAP[err.code] ?? 500; - res.status(status).json({ - type: `https://docs.example.com/errors/${err.code.toLowerCase()}`, - title: err.code.replace(/_/g, " ").toLowerCase(), - status, - detail: err.message, - ...err.details, - }); - return; - } - - // Unhandled errors - log full details, return generic message - console.error("Unhandled error:", err); - res.status(500).json({ - type: "https://docs.example.com/errors/internal", - title: "Internal Server Error", - status: 500, - detail: "An unexpected error occurred.", - }); -} - -// Register as the last middleware -app.use(errorHandler); -``` - ---- - -### 6. Error Logging Integration - -Attach structured context to errors so they are searchable and actionable in observability tools. - -**Python - Structured error logging** - -```python -import logging -import traceback -from contextvars import ContextVar - -request_id_var: ContextVar[str] = ContextVar("request_id", default="unknown") - - -class StructuredErrorLogger: - """Wraps the standard logger to attach error context automatically.""" - - def __init__(self, name: str) -> None: - self.logger = logging.getLogger(name) - - def error( - self, - msg: str, - *, - exc: Exception | None = None, - extra: dict | None = None, - ) -> None: - context = { - "request_id": request_id_var.get(), - **(extra or {}), - } - - if exc is not None: - context["error_type"] = type(exc).__name__ - context["error_message"] = str(exc) - context["stacktrace"] = traceback.format_exception(exc) - - if isinstance(exc, AppError): - context["error_code"] = exc.code.value - context["error_details"] = exc.details - - self.logger.error(msg, extra={"structured": context}, exc_info=exc) - - -# Usage in a FastAPI middleware -from starlette.middleware.base import BaseHTTPMiddleware -import uuid - -log = StructuredErrorLogger(__name__) - - -class ErrorLoggingMiddleware(BaseHTTPMiddleware): - async def dispatch(self, request, call_next): - rid = request.headers.get("x-request-id", str(uuid.uuid4())) - request_id_var.set(rid) - try: - response = await call_next(request) - return response - except Exception as exc: - log.error( - "Request failed", - exc=exc, - extra={"method": request.method, "path": request.url.path}, - ) - raise -``` - -**TypeScript - Error context enrichment** - -```typescript -interface ErrorContext { - requestId?: string; - userId?: string; - operation?: string; - [key: string]: unknown; -} - -function logError( - message: string, - error: unknown, - context: ErrorContext = {} -): void { - const payload: Record = { - message, - timestamp: new Date().toISOString(), - ...context, - }; - - if (error instanceof AppError) { - payload.errorCode = error.code; - payload.errorMessage = error.message; - payload.errorDetails = error.details; - } else if (error instanceof Error) { - payload.errorType = error.name; - payload.errorMessage = error.message; - payload.stack = error.stack; - } else { - payload.errorRaw = String(error); - } - - // Structured JSON log for ingestion by Datadog, CloudWatch, etc. - console.error(JSON.stringify(payload)); -} - -// Usage -try { - await processOrder(orderId); -} catch (error) { - logError("Order processing failed", error, { - requestId: req.headers["x-request-id"], - operation: "processOrder", - orderId, - }); - throw error; -} -``` - ---- - -### 7. Result Pattern - -Use a Result type for operations where failure is an expected outcome. Avoids exception overhead and makes the failure path explicit in the type signature. - -**Python - Using the `result` library** - -```python -# pip install result -from result import Ok, Err, Result - - -def parse_age(value: str) -> Result[int, str]: - """Parse a string to a valid age. Returns Err for invalid input.""" - try: - age = int(value) - except ValueError: - return Err(f"'{value}' is not a number") - - if age < 0 or age > 150: - return Err(f"Age {age} is out of valid range (0-150)") - - return Ok(age) - - -def validate_registration(data: dict) -> Result[dict, list[str]]: - """Validate all fields, collecting every error instead of failing on the first.""" - errors: list[str] = [] - - match parse_age(data.get("age", "")): - case Ok(age): - data["age"] = age - case Err(msg): - errors.append(msg) - - name = data.get("name", "").strip() - if not name: - errors.append("Name is required") - if len(name) > 100: - errors.append("Name must be 100 characters or fewer") - - if errors: - return Err(errors) - return Ok(data) - - -# Caller handles both paths explicitly -match validate_registration(form_data): - case Ok(valid): - user = create_user(valid) - case Err(errs): - return {"errors": errs}, 400 -``` - -**TypeScript - Discriminated union Result** - -```typescript -type Result = - | { ok: true; value: T } - | { ok: false; error: E }; - -function ok(value: T): Result { - return { ok: true, value }; -} - -function err(error: E): Result { - return { ok: false, error }; -} - -// Usage -function parseAge(input: string): Result { - const age = Number(input); - if (Number.isNaN(age)) return err(`'${input}' is not a number`); - if (age < 0 || age > 150) return err(`Age ${age} out of range (0-150)`); - return ok(age); -} - -function validateRegistration( - data: Record -): Result<{ name: string; age: number }, string[]> { - const errors: string[] = []; - - const ageResult = parseAge(data.age ?? ""); - if (!ageResult.ok) errors.push(ageResult.error); - - const name = data.name?.trim() ?? ""; - if (!name) errors.push("Name is required"); - if (name.length > 100) errors.push("Name must be 100 characters or fewer"); - - if (errors.length > 0) return err(errors); - return ok({ name, age: (ageResult as { ok: true; value: number }).value }); -} - -// Caller -const result = validateRegistration(formData); -if (!result.ok) { - return res.status(400).json({ errors: result.error }); -} -const user = await createUser(result.value); -``` - ---- - -## Best Practices - -1. **Catch specific exceptions, not bare `except` or `catch`.** A catch-all hides bugs. Catch only the errors you know how to handle and let everything else propagate. - -2. **Translate errors at architectural boundaries.** A database `IntegrityError` should become a domain `DuplicateEntryError` at the repository layer, then an HTTP 409 at the API layer. Each layer speaks its own error language. - -3. **Preserve the original cause.** Always chain the original exception (`raise X from original` in Python, `{ cause }` in TypeScript) so the root cause is visible in logs and debuggers. - -4. **Fail fast, recover high.** Detect errors as early as possible (validate inputs at the boundary) but handle them at the highest level that has enough context to decide what to do (e.g., return an HTTP response, show a fallback UI). - -5. **Never swallow errors silently.** An empty `except: pass` or `catch {}` is almost always a bug. At minimum, log the error. If you intentionally ignore it, leave a comment explaining why. - -6. **Use the Result pattern for expected failures.** When a function can legitimately fail (parsing, validation, lookups), return a Result instead of throwing. Reserve exceptions for truly unexpected situations. - -7. **Make errors actionable.** Every error message should help the reader fix the problem. Include what happened, what was expected, and what the caller can do about it. `"User not found"` is worse than `"User with id '123' not found. Verify the id and check that the user has not been deleted."`. - -8. **Test the error paths.** Write explicit tests for every error branch. Verify the error type, message, and status code. Error paths that are never tested are error paths that will break in production. - ---- - -## Common Pitfalls - -1. **Catching too broadly.** Using `except Exception` or `catch (e: any)` silences programming errors like `TypeError` or `ReferenceError` that should crash loudly during development. - -2. **Logging and re-throwing without deduplication.** If every layer logs the same error, you get five log entries for one failure. Log at the outermost handler and let inner layers propagate. - -3. **Returning error data in the wrong shape.** Mixing `{ error: "..." }`, `{ message: "..." }`, and `{ errors: [...] }` across endpoints forces every client to handle multiple formats. Pick one shape and enforce it globally. - -4. **Leaking internal details to clients.** Stack traces, database table names, and file paths in API responses are a security risk. Sanitize errors before they leave the server. - -5. **Retrying non-idempotent operations.** Retrying a `POST /orders` that partially succeeded can create duplicate orders. Only retry operations that are safe to repeat, or use idempotency keys. - -6. **Ignoring async error boundaries.** In React, error boundaries do not catch errors inside event handlers or async callbacks. Use try/catch inside `onClick`, `useEffect` cleanup, and promise chains separately. - ---- - -## Related Skills - -- `patterns/logging` - Structured logging setup and conventions -- `patterns/api-client` - HTTP client wrappers with built-in error handling -- `security/owasp` - Preventing information leakage through error messages -- `languages/python` - Python exception syntax and idioms -- `languages/typescript` - TypeScript error types and narrowing diff --git a/.claude/skills/patterns/logging/SKILL.md b/.claude/skills/patterns/logging/SKILL.md deleted file mode 100644 index fd33703..0000000 --- a/.claude/skills/patterns/logging/SKILL.md +++ /dev/null @@ -1,962 +0,0 @@ ---- -name: logging -description: > - Structured logging patterns for Python and Node.js applications. Use this skill when setting up loggers, choosing log levels, implementing correlation IDs for request tracing, redacting sensitive data from logs, or configuring log aggregation. Trigger whenever code uses console.log, print(), logging module, winston, pino, structlog, or any logging library. Also applies when building observability, debugging production issues, or adding telemetry. ---- - -# Logging - -## When to Use - -- Setting up structured logging in a new application or service -- Replacing `console.log` or `print()` with proper logging infrastructure -- Adding request tracing with correlation IDs across microservices -- Redacting sensitive data (passwords, tokens, PII) from log output -- Building observability pipelines with log aggregation (ELK, Datadog, CloudWatch) - -## When NOT to Use - -- Static analysis or linting tasks that do not involve runtime output -- Pure computation functions where logging would add unnecessary noise -- Test assertions — use testing frameworks' built-in assertion messages, not log output - ---- - -## Core Patterns - -### 1. Structured Logging Setup - -Structured logging outputs machine-parseable JSON instead of free-form strings. This enables searching, filtering, and alerting in log aggregation systems. - -#### Python with structlog - -```python -# logging_config.py -import logging -import structlog - -def configure_logging(log_level: str = "INFO", json_output: bool = True) -> None: - """Configure structured logging for the application. - - Call this once at application startup, before any loggers are created. - """ - # Set the stdlib logging level as the baseline filter - logging.basicConfig( - format="%(message)s", - level=getattr(logging, log_level.upper()), - ) - - # Choose renderers based on environment - if json_output: - renderer = structlog.processors.JSONRenderer() - else: - # Human-readable output for local development - renderer = structlog.dev.ConsoleRenderer(colors=True) - - structlog.configure( - processors=[ - structlog.contextvars.merge_contextvars, - structlog.stdlib.filter_by_level, - structlog.stdlib.add_logger_name, - structlog.stdlib.add_log_level, - structlog.processors.TimeStamper(fmt="iso"), - structlog.processors.StackInfoRenderer(), - structlog.processors.format_exc_info, - structlog.processors.UnicodeDecoder(), - renderer, - ], - context_class=dict, - logger_factory=structlog.stdlib.LoggerFactory(), - wrapper_class=structlog.stdlib.BoundLogger, - cache_logger_on_first_use=True, - ) -``` - -```python -# Usage anywhere in the application -import structlog - -logger = structlog.get_logger(__name__) - -async def create_user(email: str) -> User: - logger.info("creating_user", email=email) - user = await user_repo.create(email=email) - logger.info("user_created", user_id=user.id, email=email) - return user -``` - -**Output (JSON mode):** -```json -{"event": "user_created", "user_id": 42, "email": "alice@example.com", "logger": "app.services.user", "level": "info", "timestamp": "2025-06-15T10:30:00.123Z"} -``` - -#### Node.js with pino - -```typescript -// logger.ts -import pino from "pino"; - -export const logger = pino({ - level: process.env.LOG_LEVEL ?? "info", - // Use pretty printing only in development - transport: - process.env.NODE_ENV === "development" - ? { target: "pino-pretty", options: { colorize: true } } - : undefined, - // Base fields included in every log line - base: { - service: process.env.SERVICE_NAME ?? "api", - version: process.env.APP_VERSION ?? "unknown", - }, - // Customize serialization - serializers: { - err: pino.stdSerializers.err, - req: pino.stdSerializers.req, - res: pino.stdSerializers.res, - }, - // Redact sensitive fields (see Pattern 4) - redact: ["req.headers.authorization", "req.headers.cookie"], -}); - -// Create child loggers for specific modules -export function createLogger(module: string): pino.Logger { - return logger.child({ module }); -} -``` - -```typescript -// Usage in a service -import { createLogger } from "./logger"; - -const log = createLogger("user-service"); - -export async function createUser(email: string): Promise { - log.info({ email }, "creating_user"); - const user = await userRepo.create({ email }); - log.info({ userId: user.id, email }, "user_created"); - return user; -} -``` - -**Output (JSON):** -```json -{"level":30,"time":1718444400123,"service":"api","module":"user-service","userId":42,"email":"alice@example.com","msg":"user_created"} -``` - -### 2. Log Levels - -Use log levels consistently to control verbosity and enable filtering in production. - -| Level | When to Use | Example | -|-------|-------------|---------| -| `DEBUG` | Detailed diagnostic information useful only during development or debugging | Variable values, SQL queries, cache hits/misses | -| `INFO` | Confirmation that things are working as expected | Request received, user created, job completed | -| `WARNING` | Something unexpected happened but the application can continue | Deprecated API called, retry attempt, approaching rate limit | -| `ERROR` | A specific operation failed but the application continues running | Database query failed, external API returned 500, payment declined | -| `CRITICAL` | The application cannot continue or is in an unrecoverable state | Database connection pool exhausted, out of disk space, configuration missing | - -#### Python examples - -```python -import structlog - -logger = structlog.get_logger(__name__) - -# DEBUG: Detailed internals for troubleshooting -logger.debug("cache_lookup", key="user:42", hit=True, ttl_remaining=120) - -# INFO: Normal business events -logger.info("order_placed", order_id="ORD-123", total=99.99, items=3) - -# WARNING: Degraded but functional -logger.warning( - "rate_limit_approaching", - current_rate=450, - limit=500, - window_seconds=60, -) - -# ERROR: Operation failed, needs attention -logger.error( - "payment_failed", - order_id="ORD-123", - provider="stripe", - error_code="card_declined", - exc_info=True, # Include stack trace -) - -# CRITICAL: System-level failure -logger.critical( - "database_pool_exhausted", - active_connections=100, - max_connections=100, - waiting_requests=47, -) -``` - -#### TypeScript examples - -```typescript -import { createLogger } from "./logger"; - -const log = createLogger("order-service"); - -// DEBUG: Internal details -log.debug({ key: "user:42", hit: true, ttlRemaining: 120 }, "cache_lookup"); - -// INFO: Normal events -log.info({ orderId: "ORD-123", total: 99.99, items: 3 }, "order_placed"); - -// WARNING: Degraded state -log.warn( - { currentRate: 450, limit: 500, windowSeconds: 60 }, - "rate_limit_approaching" -); - -// ERROR: Operation failure -log.error( - { orderId: "ORD-123", provider: "stripe", errorCode: "card_declined" }, - "payment_failed" -); - -// FATAL: Unrecoverable -log.fatal( - { activeConnections: 100, maxConnections: 100, waitingRequests: 47 }, - "database_pool_exhausted" -); -``` - -**Level selection rule of thumb:** If you would page someone at 3 AM, it is ERROR or CRITICAL. If it is useful context for investigating an issue, it is INFO. If it is only useful when actively debugging a specific problem, it is DEBUG. - -### 3. Correlation IDs - -Correlation IDs (also called request IDs or trace IDs) tie together all log entries from a single request as it flows through your system. - -#### Python — FastAPI middleware with contextvars - -```python -# middleware/correlation.py -import uuid -from contextvars import ContextVar - -import structlog -from starlette.middleware.base import BaseHTTPMiddleware -from starlette.requests import Request - -# Context variable accessible from any async task in the same request -correlation_id_var: ContextVar[str] = ContextVar("correlation_id", default="") - -class CorrelationIDMiddleware(BaseHTTPMiddleware): - async def dispatch(self, request: Request, call_next): - # Accept an incoming correlation ID or generate a new one - correlation_id = request.headers.get("X-Correlation-ID", uuid.uuid4().hex) - correlation_id_var.set(correlation_id) - - # Bind to structlog context so all logs in this request include it - structlog.contextvars.clear_contextvars() - structlog.contextvars.bind_contextvars(correlation_id=correlation_id) - - response = await call_next(request) - response.headers["X-Correlation-ID"] = correlation_id - return response -``` - -```python -# Register the middleware -from middleware.correlation import CorrelationIDMiddleware - -app.add_middleware(CorrelationIDMiddleware) -``` - -```python -# Any logger call in any module now includes correlation_id automatically -logger = structlog.get_logger(__name__) - -async def get_user(user_id: int) -> User: - logger.info("fetching_user", user_id=user_id) - # Output: {"event": "fetching_user", "user_id": 42, "correlation_id": "a1b2c3d4...", ...} - return await user_repo.get(user_id) -``` - -#### TypeScript — Express middleware with AsyncLocalStorage - -```typescript -// middleware/correlation.ts -import { AsyncLocalStorage } from "node:async_hooks"; -import { randomUUID } from "node:crypto"; -import type { Request, Response, NextFunction } from "express"; -import { logger } from "../logger"; - -interface RequestContext { - correlationId: string; -} - -export const asyncLocalStorage = new AsyncLocalStorage(); - -export function correlationMiddleware( - req: Request, - res: Response, - next: NextFunction -): void { - const correlationId = - (req.headers["x-correlation-id"] as string) ?? randomUUID(); - - res.setHeader("X-Correlation-ID", correlationId); - - asyncLocalStorage.run({ correlationId }, () => { - next(); - }); -} -``` - -```typescript -// logger.ts — augment the logger to include correlation ID -import { asyncLocalStorage } from "./middleware/correlation"; - -export function getContextLogger(): pino.Logger { - const store = asyncLocalStorage.getStore(); - if (store) { - return logger.child({ correlationId: store.correlationId }); - } - return logger; -} -``` - -```typescript -// Usage in any module -import { getContextLogger } from "./logger"; - -export async function getUser(userId: number): Promise { - const log = getContextLogger(); - log.info({ userId }, "fetching_user"); - // Output includes correlationId automatically - return await userRepo.findById(userId); -} -``` - -#### Propagating to downstream services - -When calling other microservices, forward the correlation ID: - -```python -# Python — httpx client -import httpx -from middleware.correlation import correlation_id_var - -async def call_billing_service(user_id: int) -> dict: - correlation_id = correlation_id_var.get() - async with httpx.AsyncClient() as client: - response = await client.get( - f"http://billing-service/api/v1/invoices?user_id={user_id}", - headers={"X-Correlation-ID": correlation_id}, - ) - return response.json() -``` - -```typescript -// TypeScript — fetch with correlation ID -import { asyncLocalStorage } from "./middleware/correlation"; - -export async function callBillingService(userId: number): Promise { - const store = asyncLocalStorage.getStore(); - const response = await fetch( - `http://billing-service/api/v1/invoices?user_id=${userId}`, - { - headers: { - "X-Correlation-ID": store?.correlationId ?? "", - }, - } - ); - return response.json(); -} -``` - -### 4. Sensitive Data Redaction - -Never log passwords, API keys, tokens, credit card numbers, or personally identifiable information (PII). Build redaction into the logging pipeline so developers cannot accidentally leak secrets. - -#### Python — structlog processor - -```python -# processors/redact.py -import re -from typing import Any - -# Patterns for sensitive field names (case-insensitive matching) -SENSITIVE_KEYS = re.compile( - r"(password|passwd|secret|token|api_key|apikey|authorization|" - r"credit_card|card_number|cvv|ssn|social_security)", - re.IGNORECASE, -) - -# Pattern for credit card numbers in string values -CREDIT_CARD_PATTERN = re.compile(r"\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b") - -# Pattern for bearer tokens in string values -BEARER_PATTERN = re.compile(r"Bearer\s+[A-Za-z0-9\-._~+/]+=*", re.IGNORECASE) - - -def redact_sensitive_data( - logger: Any, method_name: str, event_dict: dict -) -> dict: - """Structlog processor that masks sensitive values.""" - return _redact_dict(event_dict) - - -def _redact_dict(data: dict) -> dict: - result = {} - for key, value in data.items(): - if SENSITIVE_KEYS.search(key): - result[key] = "***REDACTED***" - elif isinstance(value, dict): - result[key] = _redact_dict(value) - elif isinstance(value, str): - result[key] = _redact_string(value) - elif isinstance(value, list): - result[key] = [ - _redact_dict(item) if isinstance(item, dict) else item - for item in value - ] - else: - result[key] = value - return result - - -def _redact_string(value: str) -> str: - value = CREDIT_CARD_PATTERN.sub("****-****-****-****", value) - value = BEARER_PATTERN.sub("Bearer ***REDACTED***", value) - return value -``` - -```python -# Add the processor to structlog configuration -structlog.configure( - processors=[ - structlog.contextvars.merge_contextvars, - redact_sensitive_data, # Add before the renderer - structlog.processors.JSONRenderer(), - ], - # ... -) -``` - -#### TypeScript — pino redaction - -```typescript -// pino has built-in redaction support -import pino from "pino"; - -export const logger = pino({ - level: "info", - redact: { - paths: [ - "password", - "secret", - "token", - "apiKey", - "authorization", - "creditCard", - "req.headers.authorization", - "req.headers.cookie", - "body.password", - "body.creditCardNumber", - "*.password", - "*.secret", - ], - censor: "***REDACTED***", - }, -}); -``` - -For more complex redaction (regex-based), use a custom serializer: - -```typescript -// redact.ts -const CREDIT_CARD_RE = /\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g; -const BEARER_RE = /Bearer\s+[A-Za-z0-9\-._~+/]+=*/gi; - -export function redactValue(value: unknown): unknown { - if (typeof value === "string") { - let result = value.replace(CREDIT_CARD_RE, "****-****-****-****"); - result = result.replace(BEARER_RE, "Bearer ***REDACTED***"); - return result; - } - if (typeof value === "object" && value !== null) { - return redactObject(value as Record); - } - return value; -} - -const SENSITIVE_KEYS = - /^(password|passwd|secret|token|api_?key|authorization|credit_?card|cvv|ssn)$/i; - -function redactObject(obj: Record): Record { - const result: Record = {}; - for (const [key, value] of Object.entries(obj)) { - if (SENSITIVE_KEYS.test(key)) { - result[key] = "***REDACTED***"; - } else { - result[key] = redactValue(value); - } - } - return result; -} -``` - -### 5. Request/Response Logging - -Log every HTTP request and response with method, path, status code, duration, and body size. This is the single most valuable log line for production debugging. - -#### Python — FastAPI middleware - -```python -# middleware/request_logging.py -import time -import structlog -from starlette.middleware.base import BaseHTTPMiddleware -from starlette.requests import Request - -logger = structlog.get_logger("http") - -class RequestLoggingMiddleware(BaseHTTPMiddleware): - async def dispatch(self, request: Request, call_next): - start_time = time.perf_counter() - - # Log request - logger.info( - "http_request_started", - method=request.method, - path=request.url.path, - query=str(request.url.query) or None, - client_ip=request.client.host if request.client else None, - user_agent=request.headers.get("user-agent"), - ) - - try: - response = await call_next(request) - except Exception: - duration_ms = (time.perf_counter() - start_time) * 1000 - logger.error( - "http_request_failed", - method=request.method, - path=request.url.path, - duration_ms=round(duration_ms, 2), - exc_info=True, - ) - raise - - duration_ms = (time.perf_counter() - start_time) * 1000 - content_length = response.headers.get("content-length") - - # Choose log level based on status code - log_method = logger.info - if response.status_code >= 500: - log_method = logger.error - elif response.status_code >= 400: - log_method = logger.warning - - log_method( - "http_request_completed", - method=request.method, - path=request.url.path, - status_code=response.status_code, - duration_ms=round(duration_ms, 2), - content_length=int(content_length) if content_length else None, - ) - - return response -``` - -#### TypeScript — Express middleware - -```typescript -// middleware/request-logging.ts -import type { Request, Response, NextFunction } from "express"; -import { getContextLogger } from "../logger"; - -export function requestLogging( - req: Request, - res: Response, - next: NextFunction -): void { - const start = process.hrtime.bigint(); - const log = getContextLogger(); - - log.info( - { - method: req.method, - path: req.originalUrl, - clientIp: req.ip, - userAgent: req.get("user-agent"), - }, - "http_request_started" - ); - - // Hook into the response finish event - res.on("finish", () => { - const durationMs = - Number(process.hrtime.bigint() - start) / 1_000_000; - - const logFn = - res.statusCode >= 500 - ? log.error.bind(log) - : res.statusCode >= 400 - ? log.warn.bind(log) - : log.info.bind(log); - - logFn( - { - method: req.method, - path: req.originalUrl, - statusCode: res.statusCode, - durationMs: Math.round(durationMs * 100) / 100, - contentLength: res.get("content-length") - ? parseInt(res.get("content-length")!, 10) - : undefined, - }, - "http_request_completed" - ); - }); - - next(); -} -``` - -```typescript -// Register middleware (order matters) -app.use(correlationMiddleware); -app.use(requestLogging); -``` - -### 6. Error Logging - -When logging errors, always include the stack trace, relevant context, and enough information to reproduce the issue without accessing the production environment. - -#### Python — exception logging with context - -```python -import structlog - -logger = structlog.get_logger(__name__) - -async def process_order(order_id: str) -> Order: - logger.info("processing_order", order_id=order_id) - - try: - order = await order_repo.get(order_id) - if not order: - logger.error("order_not_found", order_id=order_id) - raise OrderNotFoundError(order_id) - - payment = await payment_service.charge( - amount=order.total, - currency=order.currency, - customer_id=order.customer_id, - ) - logger.info( - "payment_processed", - order_id=order_id, - payment_id=payment.id, - amount=order.total, - ) - - except PaymentError as exc: - # Log the error with full context for debugging - logger.error( - "payment_failed", - order_id=order_id, - customer_id=order.customer_id, - amount=order.total, - error_code=exc.code, - error_message=str(exc), - exc_info=True, # Includes full stack trace - ) - raise - - except Exception as exc: - # Catch-all for unexpected errors - logger.exception( - "order_processing_unexpected_error", - order_id=order_id, - error_type=type(exc).__name__, - ) - raise -``` - -#### TypeScript — error logging with breadcrumbs - -```typescript -import { getContextLogger } from "./logger"; - -const log = getContextLogger(); - -export async function processOrder(orderId: string): Promise { - log.info({ orderId }, "processing_order"); - - try { - const order = await orderRepo.findById(orderId); - if (!order) { - log.error({ orderId }, "order_not_found"); - throw new OrderNotFoundError(orderId); - } - - const payment = await paymentService.charge({ - amount: order.total, - currency: order.currency, - customerId: order.customerId, - }); - - log.info( - { orderId, paymentId: payment.id, amount: order.total }, - "payment_processed" - ); - return order; - } catch (err) { - if (err instanceof PaymentError) { - log.error( - { - orderId, - errorCode: err.code, - errorMessage: err.message, - err, // pino serializes Error objects with stack traces - }, - "payment_failed" - ); - } else { - log.error( - { - orderId, - err, - errorType: (err as Error).constructor.name, - }, - "order_processing_unexpected_error" - ); - } - throw err; - } -} -``` - -**Key principle:** Log at the point where you have the most context, not at every layer. A single error log with full context is more useful than five partial logs scattered across the call stack. - -### 7. Performance Logging - -Track operation durations to identify slow endpoints, queries, and external calls. - -#### Python — timing decorator - -```python -import functools -import time -from typing import Callable, TypeVar - -import structlog - -logger = structlog.get_logger("performance") - -F = TypeVar("F", bound=Callable) - -def log_duration(operation: str, slow_threshold_ms: float = 1000.0): - """Decorator that logs the duration of a function call. - - Args: - operation: A descriptive name for the operation. - slow_threshold_ms: Threshold in milliseconds above which - the log level escalates to WARNING. - """ - def decorator(func: F) -> F: - @functools.wraps(func) - async def async_wrapper(*args, **kwargs): - start = time.perf_counter() - try: - result = await func(*args, **kwargs) - return result - finally: - duration_ms = (time.perf_counter() - start) * 1000 - log_fn = ( - logger.warning - if duration_ms > slow_threshold_ms - else logger.debug - ) - log_fn( - "operation_duration", - operation=operation, - duration_ms=round(duration_ms, 2), - slow=duration_ms > slow_threshold_ms, - ) - - @functools.wraps(func) - def sync_wrapper(*args, **kwargs): - start = time.perf_counter() - try: - result = func(*args, **kwargs) - return result - finally: - duration_ms = (time.perf_counter() - start) * 1000 - log_fn = ( - logger.warning - if duration_ms > slow_threshold_ms - else logger.debug - ) - log_fn( - "operation_duration", - operation=operation, - duration_ms=round(duration_ms, 2), - slow=duration_ms > slow_threshold_ms, - ) - - import asyncio - if asyncio.iscoroutinefunction(func): - return async_wrapper # type: ignore - return sync_wrapper # type: ignore - - return decorator - - -# Usage -@log_duration("fetch_user_profile", slow_threshold_ms=200) -async def get_user_profile(user_id: int) -> UserProfile: - return await user_repo.get_with_preferences(user_id) -``` - -#### Python — context manager for ad-hoc timing - -```python -import time -from contextlib import contextmanager - -import structlog - -logger = structlog.get_logger("performance") - -@contextmanager -def log_timing(operation: str, **extra_fields): - """Context manager for timing arbitrary code blocks.""" - start = time.perf_counter() - yield - duration_ms = (time.perf_counter() - start) * 1000 - logger.info( - "operation_duration", - operation=operation, - duration_ms=round(duration_ms, 2), - **extra_fields, - ) - -# Usage -async def rebuild_search_index(): - with log_timing("rebuild_search_index", index="products"): - products = await product_repo.get_all() - await search_service.reindex(products) -``` - -#### TypeScript — timing wrapper - -```typescript -import { createLogger } from "./logger"; - -const perfLog = createLogger("performance"); - -export async function withTiming( - operation: string, - fn: () => Promise, - slowThresholdMs = 1000 -): Promise { - const start = performance.now(); - try { - const result = await fn(); - return result; - } finally { - const durationMs = performance.now() - start; - const logFn = durationMs > slowThresholdMs ? perfLog.warn : perfLog.debug; - logFn( - { - operation, - durationMs: Math.round(durationMs * 100) / 100, - slow: durationMs > slowThresholdMs, - }, - "operation_duration" - ); - } -} - -// Usage -const profile = await withTiming("fetch_user_profile", () => - userRepo.getWithPreferences(userId) -); -``` - -#### Slow query logging - -```python -# Python — SQLAlchemy event listener for slow queries -from sqlalchemy import event -from sqlalchemy.engine import Engine - -import structlog - -logger = structlog.get_logger("database") - -SLOW_QUERY_THRESHOLD_MS = 500 - -@event.listens_for(Engine, "before_cursor_execute") -def before_cursor_execute(conn, cursor, statement, parameters, context, executemany): - conn.info.setdefault("query_start_time", []).append(time.perf_counter()) - -@event.listens_for(Engine, "after_cursor_execute") -def after_cursor_execute(conn, cursor, statement, parameters, context, executemany): - total_ms = (time.perf_counter() - conn.info["query_start_time"].pop()) * 1000 - if total_ms > SLOW_QUERY_THRESHOLD_MS: - logger.warning( - "slow_query", - duration_ms=round(total_ms, 2), - statement=statement[:500], # Truncate long queries - parameters=str(parameters)[:200], - ) -``` - ---- - -## Best Practices - -1. **Use structured logging from day one** — start with JSON output and key-value pairs instead of formatted strings. Switching from `f"User {user_id} created"` to `logger.info("user_created", user_id=user_id)` costs nothing upfront but saves hours when debugging in production. - -2. **Log events, not sentences** — use snake_case event names (`order_placed`, `payment_failed`) rather than prose messages (`"An order was placed by the user"`). Event names are searchable, filterable, and easy to aggregate. - -3. **Include the right context at the right level** — every log line should include enough context to be useful in isolation: relevant IDs (user, order, request), operation name, and outcome. Avoid logging the same error at every layer of the call stack. - -4. **Set log levels per environment** — use DEBUG in development, INFO in staging, and INFO or WARNING in production. Never leave DEBUG enabled in production — it generates excessive volume and may expose sensitive internals. - -5. **Centralize logging configuration** — configure loggers once at application startup, not in individual modules. Every module should call `get_logger(__name__)` and inherit the shared configuration. - -6. **Always redact sensitive data** — build redaction into the logging pipeline as a processor or serializer. Do not rely on developers remembering to exclude passwords or tokens from log calls. - -7. **Use correlation IDs for every request** — generate a unique ID at the entry point and propagate it through all downstream calls. This is the single most important pattern for debugging distributed systems. - -8. **Set up log rotation and retention policies** — configure maximum file sizes, rotation intervals, and retention periods. Production logs without rotation will fill disks. Use log aggregation services (ELK, Datadog, CloudWatch) rather than relying on local files. - ---- - -## Common Pitfalls - -1. **Logging sensitive data** — passwords, API keys, JWTs, credit card numbers, and PII end up in logs more often than expected. Once written, they persist in log storage and backups. Build redaction into the pipeline rather than relying on code review to catch every instance. - -2. **Using print() or console.log in production** — `print()` in Python and `console.log` in Node.js write to stdout without timestamps, levels, or structure. They cannot be filtered, aggregated, or searched. Replace them with a proper logger before deploying. - -3. **Logging too much at high levels** — setting every log call to INFO or ERROR creates alert fatigue and obscures real problems. Use DEBUG for diagnostic details and reserve ERROR for situations that require action. - -4. **Missing stack traces on errors** — logging `str(exception)` loses the stack trace. In Python, use `exc_info=True` or `logger.exception()`. In pino, pass the error as `{ err }` to get the full stack serialized. - -5. **Not testing log output** — logging code is code. If your redaction processor has a bug, secrets leak. Write unit tests that capture log output and assert on structure, redacted fields, and expected context. - -6. **Synchronous logging in async applications** — writing to files or network sinks synchronously from an async event loop blocks request processing. Use async-compatible transports (pino's worker thread, structlog with stdlib async handlers) or write to stdout and let the infrastructure handle routing. - ---- - -## Related Skills - -- `patterns/error-handling` — Exception handling patterns that complement error logging -- `patterns/api-client` — HTTP client patterns including logging outbound requests -- `frameworks/fastapi` — FastAPI middleware setup for request logging and correlation IDs -- `devops/docker` — Container logging drivers and log aggregation in Docker environments -- `databases/postgresql` — Logging database queries and slow query detection -- `databases/mongodb` — Logging database operations and aggregation pipelines diff --git a/.claude/skills/playwright/SKILL.md b/.claude/skills/playwright/SKILL.md new file mode 100644 index 0000000..9e3b347 --- /dev/null +++ b/.claude/skills/playwright/SKILL.md @@ -0,0 +1,423 @@ +--- +name: playwright +description: Use when writing, debugging, or configuring E2E tests with Playwright. Trigger for any mention of end-to-end testing, browser automation, page objects, visual regression, storageState auth, playwright.config, or cross-browser testing. Also use when setting up E2E in CI, testing critical user flows, or debugging flaky browser tests. +--- + +# Playwright E2E Testing + +## Overview + +The definitive E2E testing reference for web apps built with Next.js, FastAPI, Django, NestJS, Express, and React. Covers test structure, locator strategy, authentication reuse, API mocking, visual regression, accessibility, CI sharding, and framework-specific setup. + +## When to Use +- Testing critical user flows end-to-end (login, checkout, onboarding) +- Cross-browser testing (Chromium, Firefox, WebKit) +- Visual regression testing with `toHaveScreenshot()` +- Accessibility auditing with `@axe-core/playwright` +- Testing Server Components, SSR pages, or full-stack flows +- Mobile/responsive testing via device emulation + +## When NOT to Use +- **Unit testing** isolated functions — use `pytest` or `vitest` +- **Component testing** React components in isolation — use `vitest` + Testing Library (faster feedback loop) +- **API-only testing** with no browser interaction — use `httpx` / `supertest` directly +- **Load/performance testing** — use k6, Artillery, or Locust + +--- + +## Quick Reference + +| I need... | Go to | +|-----------|-------| +| Production-grade config to copy | [templates/playwright.config.ts](templates/playwright.config.ts) | +| Page Object, auth, mocking patterns | [references/e2e-patterns.md](references/e2e-patterns.md) | +| Locator strategy | § Locators below | +| Auth reuse with storageState | § Authentication below | +| CI setup (GitHub Actions + sharding) | § CI Integration below | +| Framework-specific webServer | § Framework Integration below | + +--- + +## Core Patterns + +### Test Structure + +```typescript +import { test, expect } from '@playwright/test'; + +test.describe('Checkout flow', () => { + test('guest can complete purchase', async ({ page }) => { + await page.goto('/products/widget-pro'); + await page.getByRole('button', { name: 'Add to cart' }).click(); + await page.getByRole('link', { name: 'Cart' }).click(); + await page.getByRole('button', { name: 'Checkout' }).click(); + + await page.getByLabel('Email').fill('guest@example.com'); + await page.getByRole('button', { name: 'Place order' }).click(); + + await expect(page.getByText('Order confirmed')).toBeVisible(); + }); +}); +``` + +### Locators — the priority order + +Always prefer **role-based and user-visible locators**. They survive refactors and match how users interact with the page. + +| Priority | Locator | When | +|----------|---------|------| +| 1 | `getByRole('button', { name: '...' })` | Interactive elements with accessible names | +| 2 | `getByLabel('...')` | Form fields with `