feat: improved the Claude Kit as a plugin

This commit is contained in:
duthaho
2026-04-19 14:09:14 +07:00
parent 3103a8da1b
commit d1a6d2a2bc
186 changed files with 771 additions and 1691 deletions
+104
View File
@@ -0,0 +1,104 @@
---
name: logging
description: >
Use when setting up loggers, choosing log levels, implementing correlation IDs for request tracing, redacting sensitive data from logs, or configuring log aggregation. Also activate whenever code uses console.log, print(), logging module, winston, pino, structlog, or any logging library. Applies when building observability, debugging production issues, or adding telemetry.
---
# Logging
## When to Use
- Setting up structured logging in a new application or service
- Replacing `console.log` or `print()` with proper logging infrastructure
- Adding request tracing with correlation IDs across microservices
- Redacting sensitive data (passwords, tokens, PII) from log output
- Building observability pipelines with log aggregation (ELK, Datadog, CloudWatch)
## When NOT to Use
- Static analysis or linting tasks that do not involve runtime output
- Pure computation functions where logging would add unnecessary noise
- Test assertions — use testing frameworks' built-in assertion messages, not log output
---
## Quick Reference
| # | Pattern | Description |
|---|---------|-------------|
| 1 | Structured Logging Setup | Configure JSON-based structured logging at application startup with environment-aware renderers |
| 2 | Log Levels | Use DEBUG/INFO/WARNING/ERROR/CRITICAL consistently to control verbosity and enable filtering |
| 3 | Correlation IDs | Generate a unique request ID at the entry point and propagate it through all downstream calls |
| 4 | Sensitive Data Redaction | Build redaction into the logging pipeline so secrets and PII are never written to logs |
| 5 | Request/Response Logging | Log every HTTP request/response with method, path, status, duration, and body size |
| 6 | Error Logging | Include stack traces, relevant IDs, and enough context to reproduce without production access |
| 7 | Performance Logging | Track operation durations to identify slow endpoints, queries, and external calls |
---
## Log Levels
| Level | When to Use | Example |
|-------|-------------|---------|
| `DEBUG` | Detailed diagnostic information useful only during development or debugging | Variable values, SQL queries, cache hits/misses |
| `INFO` | Confirmation that things are working as expected | Request received, user created, job completed |
| `WARNING` | Something unexpected happened but the application can continue | Deprecated API called, retry attempt, approaching rate limit |
| `ERROR` | A specific operation failed but the application continues running | Database query failed, external API returned 500, payment declined |
| `CRITICAL` | The application cannot continue or is in an unrecoverable state | Database connection pool exhausted, out of disk space, configuration missing |
**Level selection rule of thumb:** If you would page someone at 3 AM, it is ERROR or CRITICAL. If it is useful context for investigating an issue, it is INFO. If it is only useful when actively debugging a specific problem, it is DEBUG.
---
## Language References
See `references/python-patterns.md` for Python/structlog examples.
See `references/typescript-patterns.md` for TypeScript/pino examples.
---
## Best Practices
1. **Use structured logging from day one** — start with JSON output and key-value pairs instead of formatted strings. Switching from `f"User {user_id} created"` to `logger.info("user_created", user_id=user_id)` costs nothing upfront but saves hours when debugging in production.
2. **Log events, not sentences** — use snake_case event names (`order_placed`, `payment_failed`) rather than prose messages (`"An order was placed by the user"`). Event names are searchable, filterable, and easy to aggregate.
3. **Include the right context at the right level** — every log line should include enough context to be useful in isolation: relevant IDs (user, order, request), operation name, and outcome. Avoid logging the same error at every layer of the call stack.
4. **Set log levels per environment** — use DEBUG in development, INFO in staging, and INFO or WARNING in production. Never leave DEBUG enabled in production — it generates excessive volume and may expose sensitive internals.
5. **Centralize logging configuration** — configure loggers once at application startup, not in individual modules. Every module should call `get_logger(__name__)` and inherit the shared configuration.
6. **Always redact sensitive data** — build redaction into the logging pipeline as a processor or serializer. Do not rely on developers remembering to exclude passwords or tokens from log calls.
7. **Use correlation IDs for every request** — generate a unique ID at the entry point and propagate it through all downstream calls. This is the single most important pattern for debugging distributed systems.
8. **Set up log rotation and retention policies** — configure maximum file sizes, rotation intervals, and retention periods. Production logs without rotation will fill disks. Use log aggregation services (ELK, Datadog, CloudWatch) rather than relying on local files.
---
## Common Pitfalls
1. **Logging sensitive data** — passwords, API keys, JWTs, credit card numbers, and PII end up in logs more often than expected. Once written, they persist in log storage and backups. Build redaction into the pipeline rather than relying on code review to catch every instance.
2. **Using print() or console.log in production**`print()` in Python and `console.log` in Node.js write to stdout without timestamps, levels, or structure. They cannot be filtered, aggregated, or searched. Replace them with a proper logger before deploying.
3. **Logging too much at high levels** — setting every log call to INFO or ERROR creates alert fatigue and obscures real problems. Use DEBUG for diagnostic details and reserve ERROR for situations that require action.
4. **Missing stack traces on errors** — logging `str(exception)` loses the stack trace. In Python, use `exc_info=True` or `logger.exception()`. In pino, pass the error as `{ err }` to get the full stack serialized.
5. **Not testing log output** — logging code is code. If your redaction processor has a bug, secrets leak. Write unit tests that capture log output and assert on structure, redacted fields, and expected context.
6. **Synchronous logging in async applications** — writing to files or network sinks synchronously from an async event loop blocks request processing. Use async-compatible transports (pino's worker thread, structlog with stdlib async handlers) or write to stdout and let the infrastructure handle routing.
---
## Related Skills
- `error-handling` — Exception handling patterns that complement error logging
- `api-client` — HTTP client patterns including logging outbound requests
- `fastapi` — FastAPI middleware setup for request logging and correlation IDs
- `docker` — Container logging drivers and log aggregation in Docker environments
- `postgresql` — Logging database queries and slow query detection
- `mongodb` — Logging database operations and aggregation pipelines
+157
View File
@@ -0,0 +1,157 @@
# Log Levels Quick Reference
## Level Summary
| Level | When to Use | Audience | Production Default |
|-------|------------|----------|-------------------|
| **DEBUG** | Detailed diagnostic info | Developers debugging | Off |
| **INFO** | Routine operational events | Ops team monitoring | On |
| **WARNING** | Something unexpected but handled | Ops + Devs | On |
| **ERROR** | Operation failed, needs attention | On-call engineers | On |
| **CRITICAL** | System is unusable or data at risk | On-call + management | On + alert |
---
## DEBUG
**Purpose**: Fine-grained information useful only when diagnosing problems.
**Turn on**: During local development or when investigating a specific issue.
| Good | Bad |
|------|-----|
| `Parsing config file: /etc/app/config.yaml` | `Entering function parse_config` |
| `Cache miss for key user:123, fetching from DB` | `x = 5` |
| `SQL: SELECT * FROM users WHERE id=$1 [params: 123]` | `Here we go!` |
| `Retry attempt 2/3 for payment gateway` | `Debug debug debug` |
| `JWT token expires at 2025-01-29T10:00:00Z` | `token = eyJhbG...` (secret!) |
**Rule**: Never log secrets, tokens, passwords, or PII at any level.
---
## INFO
**Purpose**: Confirm the system is working as expected. Key business events.
| Good | Bad |
|------|-----|
| `Server started on port 8080` | `Server is running` (which port? which version?) |
| `User user:456 created account via OAuth (Google)` | `New user` |
| `Order ord:789 placed, total=$45.00, items=3` | `Order created` |
| `Migration v42 applied successfully (12 tables)` | `Migration done` |
| `Scheduled job "daily-report" completed in 4.2s` | `Job finished` |
| `Payment processed: txn:abc, amount=$99, method=card` | `Payment OK` |
**Rule**: Include enough context to answer "what happened, to what, and relevant numbers."
---
## WARNING
**Purpose**: Something unexpected happened, but the system handled it. May indicate a future problem.
| Good | Bad |
|------|-----|
| `Connection pool at 85% capacity (17/20)` | `Pool getting full` |
| `Deprecated API v1 called by client app:legacy (use v2)` | `Old API used` |
| `Disk space below 10% on /data (2.1 GB remaining)` | `Low disk` |
| `Request took 4.8s (threshold: 5s) for GET /api/search` | `Slow request` |
| `Config REDIS_URL missing, falling back to in-memory cache` | `No Redis` |
| `Rate limit approaching for IP 10.0.0.5: 90/100 requests` | `Almost rate limited` |
**Rule**: Warnings should be actionable. If nobody would investigate, it's DEBUG or INFO.
---
## ERROR
**Purpose**: An operation failed. The system can continue, but something broke.
| Good | Bad |
|------|-----|
| `Failed to send email to user:123: SMTP timeout after 30s` | `Email error` |
| `Payment declined for order:789: card_expired (Stripe)` | `Payment failed` |
| `Database query timeout after 10s: SELECT FROM orders WHERE...` | `DB error` |
| `File upload failed: S3 returned 503, bucket=media, key=img/456.jpg` | `Upload error` |
| `Unhandled exception in POST /api/orders: ValueError("...")` | (stack trace only, no context) |
**Rule**: Include the operation, the target/ID, the error detail, and what was attempted.
---
## CRITICAL
**Purpose**: System is unusable or data integrity is at risk. Requires immediate human intervention.
| Good | Bad |
|------|-----|
| `Database connection lost, all pools exhausted, 0/20 available` | `DB down` |
| `Disk full on /data, writes failing, data loss possible` | `No disk space` |
| `Security: 500 failed login attempts from IP 10.0.0.5 in 60s` | `Too many logins` |
| `Data corruption detected: order:789 total=-$50.00` | `Bad data` |
| `TLS certificate expires in 24h, auto-renewal failed` | `Cert expiring` |
**Rule**: Every CRITICAL log should trigger an alert (PagerDuty, Slack, etc.).
---
## Structured Logging Format
### Python (structlog)
```python
import structlog
log = structlog.get_logger()
log.info("order.placed", order_id="ord:789", total=45.00, items=3)
log.error("email.send_failed", user_id="user:123", error="SMTP timeout", retry=2)
```
### TypeScript (pino)
```typescript
import pino from "pino";
const log = pino({ level: "info" });
log.info({ orderId: "ord:789", total: 45.0, items: 3 }, "order.placed");
log.error({ userId: "user:123", err, retry: 2 }, "email.send_failed");
```
### Key-Value Best Practices
| Field | Purpose | Example |
|-------|---------|---------|
| `event` / message | What happened | `"order.placed"` |
| `request_id` | Trace across services | `"req_abc123"` |
| `user_id` | Who triggered it | `"user:456"` |
| `duration_ms` | How long it took | `142` |
| `error` | Error message (not stack in prod) | `"connection refused"` |
| `component` | Which module/service | `"payment-gateway"` |
---
## Configuration by Environment
| Environment | Minimum Level | Structured? | Destination |
|-------------|--------------|-------------|-------------|
| Local dev | DEBUG | No (human-readable) | stdout |
| CI/Test | WARNING | No | stdout |
| Staging | DEBUG | Yes (JSON) | Log aggregator |
| Production | INFO | Yes (JSON) | Log aggregator |
---
## Anti-Patterns
| Anti-Pattern | Problem | Fix |
|-------------|---------|-----|
| Logging PII/secrets | Security/compliance violation | Redact or mask sensitive fields |
| `log.error()` in a loop | Log flooding, storage cost | Log once with count |
| `log.error("Error: " + err)` | Missing context, hard to search | Use structured fields |
| Logging at wrong level | Alert fatigue or missed issues | Follow the guide above |
| Catch-log-rethrow | Duplicate log entries | Log at the handling site only |
| No request_id | Cannot correlate logs | Add correlation ID middleware |
| Logging full request bodies | Performance, storage, PII risk | Log summary fields only |
@@ -0,0 +1,506 @@
# Logging — Python Patterns (structlog)
Reference examples for the [logging skill](./SKILL.md). All patterns use [structlog](https://www.structlog.org/) with stdlib integration.
---
## 1. Structured Logging Setup
Configure structured logging once at application startup. All modules then use `structlog.get_logger(__name__)`.
```python
# logging_config.py
import logging
import structlog
def configure_logging(log_level: str = "INFO", json_output: bool = True) -> None:
"""Configure structured logging for the application.
Call this once at application startup, before any loggers are created.
"""
# Set the stdlib logging level as the baseline filter
logging.basicConfig(
format="%(message)s",
level=getattr(logging, log_level.upper()),
)
# Choose renderers based on environment
if json_output:
renderer = structlog.processors.JSONRenderer()
else:
# Human-readable output for local development
renderer = structlog.dev.ConsoleRenderer(colors=True)
structlog.configure(
processors=[
structlog.contextvars.merge_contextvars,
structlog.stdlib.filter_by_level,
structlog.stdlib.add_logger_name,
structlog.stdlib.add_log_level,
structlog.processors.TimeStamper(fmt="iso"),
structlog.processors.StackInfoRenderer(),
structlog.processors.format_exc_info,
structlog.processors.UnicodeDecoder(),
renderer,
],
context_class=dict,
logger_factory=structlog.stdlib.LoggerFactory(),
wrapper_class=structlog.stdlib.BoundLogger,
cache_logger_on_first_use=True,
)
```
```python
# Usage anywhere in the application
import structlog
logger = structlog.get_logger(__name__)
async def create_user(email: str) -> User:
logger.info("creating_user", email=email)
user = await user_repo.create(email=email)
logger.info("user_created", user_id=user.id, email=email)
return user
```
**Output (JSON mode):**
```json
{"event": "user_created", "user_id": 42, "email": "alice@example.com", "logger": "app.services.user", "level": "info", "timestamp": "2025-06-15T10:30:00.123Z"}
```
---
## 2. Log Levels
```python
import structlog
logger = structlog.get_logger(__name__)
# DEBUG: Detailed internals for troubleshooting
logger.debug("cache_lookup", key="user:42", hit=True, ttl_remaining=120)
# INFO: Normal business events
logger.info("order_placed", order_id="ORD-123", total=99.99, items=3)
# WARNING: Degraded but functional
logger.warning(
"rate_limit_approaching",
current_rate=450,
limit=500,
window_seconds=60,
)
# ERROR: Operation failed, needs attention
logger.error(
"payment_failed",
order_id="ORD-123",
provider="stripe",
error_code="card_declined",
exc_info=True, # Include stack trace
)
# CRITICAL: System-level failure
logger.critical(
"database_pool_exhausted",
active_connections=100,
max_connections=100,
waiting_requests=47,
)
```
---
## 3. Correlation IDs
Correlation IDs tie together all log entries from a single request. Uses FastAPI middleware with `contextvars`.
```python
# middleware/correlation.py
import uuid
from contextvars import ContextVar
import structlog
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
# Context variable accessible from any async task in the same request
correlation_id_var: ContextVar[str] = ContextVar("correlation_id", default="")
class CorrelationIDMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
# Accept an incoming correlation ID or generate a new one
correlation_id = request.headers.get("X-Correlation-ID", uuid.uuid4().hex)
correlation_id_var.set(correlation_id)
# Bind to structlog context so all logs in this request include it
structlog.contextvars.clear_contextvars()
structlog.contextvars.bind_contextvars(correlation_id=correlation_id)
response = await call_next(request)
response.headers["X-Correlation-ID"] = correlation_id
return response
```
```python
# Register the middleware
from middleware.correlation import CorrelationIDMiddleware
app.add_middleware(CorrelationIDMiddleware)
```
```python
# Any logger call in any module now includes correlation_id automatically
logger = structlog.get_logger(__name__)
async def get_user(user_id: int) -> User:
logger.info("fetching_user", user_id=user_id)
# Output: {"event": "fetching_user", "user_id": 42, "correlation_id": "a1b2c3d4...", ...}
return await user_repo.get(user_id)
```
### Propagating to downstream services
When calling other microservices, forward the correlation ID:
```python
# Python — httpx client
import httpx
from middleware.correlation import correlation_id_var
async def call_billing_service(user_id: int) -> dict:
correlation_id = correlation_id_var.get()
async with httpx.AsyncClient() as client:
response = await client.get(
f"http://billing-service/api/v1/invoices?user_id={user_id}",
headers={"X-Correlation-ID": correlation_id},
)
return response.json()
```
---
## 4. Sensitive Data Redaction
Build redaction into the logging pipeline as a structlog processor so developers cannot accidentally leak secrets.
```python
# processors/redact.py
import re
from typing import Any
# Patterns for sensitive field names (case-insensitive matching)
SENSITIVE_KEYS = re.compile(
r"(password|passwd|secret|token|api_key|apikey|authorization|"
r"credit_card|card_number|cvv|ssn|social_security)",
re.IGNORECASE,
)
# Pattern for credit card numbers in string values
CREDIT_CARD_PATTERN = re.compile(r"\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b")
# Pattern for bearer tokens in string values
BEARER_PATTERN = re.compile(r"Bearer\s+[A-Za-z0-9\-._~+/]+=*", re.IGNORECASE)
def redact_sensitive_data(
logger: Any, method_name: str, event_dict: dict
) -> dict:
"""Structlog processor that masks sensitive values."""
return _redact_dict(event_dict)
def _redact_dict(data: dict) -> dict:
result = {}
for key, value in data.items():
if SENSITIVE_KEYS.search(key):
result[key] = "***REDACTED***"
elif isinstance(value, dict):
result[key] = _redact_dict(value)
elif isinstance(value, str):
result[key] = _redact_string(value)
elif isinstance(value, list):
result[key] = [
_redact_dict(item) if isinstance(item, dict) else item
for item in value
]
else:
result[key] = value
return result
def _redact_string(value: str) -> str:
value = CREDIT_CARD_PATTERN.sub("****-****-****-****", value)
value = BEARER_PATTERN.sub("Bearer ***REDACTED***", value)
return value
```
```python
# Add the processor to structlog configuration
structlog.configure(
processors=[
structlog.contextvars.merge_contextvars,
redact_sensitive_data, # Add before the renderer
structlog.processors.JSONRenderer(),
],
# ...
)
```
---
## 5. Request/Response Logging
Log every HTTP request and response with method, path, status code, duration, and body size. Uses FastAPI middleware.
```python
# middleware/request_logging.py
import time
import structlog
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
logger = structlog.get_logger("http")
class RequestLoggingMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next):
start_time = time.perf_counter()
# Log request
logger.info(
"http_request_started",
method=request.method,
path=request.url.path,
query=str(request.url.query) or None,
client_ip=request.client.host if request.client else None,
user_agent=request.headers.get("user-agent"),
)
try:
response = await call_next(request)
except Exception:
duration_ms = (time.perf_counter() - start_time) * 1000
logger.error(
"http_request_failed",
method=request.method,
path=request.url.path,
duration_ms=round(duration_ms, 2),
exc_info=True,
)
raise
duration_ms = (time.perf_counter() - start_time) * 1000
content_length = response.headers.get("content-length")
# Choose log level based on status code
log_method = logger.info
if response.status_code >= 500:
log_method = logger.error
elif response.status_code >= 400:
log_method = logger.warning
log_method(
"http_request_completed",
method=request.method,
path=request.url.path,
status_code=response.status_code,
duration_ms=round(duration_ms, 2),
content_length=int(content_length) if content_length else None,
)
return response
```
---
## 6. Error Logging
When logging errors, include the stack trace, relevant context, and enough information to reproduce the issue.
```python
import structlog
logger = structlog.get_logger(__name__)
async def process_order(order_id: str) -> Order:
logger.info("processing_order", order_id=order_id)
try:
order = await order_repo.get(order_id)
if not order:
logger.error("order_not_found", order_id=order_id)
raise OrderNotFoundError(order_id)
payment = await payment_service.charge(
amount=order.total,
currency=order.currency,
customer_id=order.customer_id,
)
logger.info(
"payment_processed",
order_id=order_id,
payment_id=payment.id,
amount=order.total,
)
except PaymentError as exc:
# Log the error with full context for debugging
logger.error(
"payment_failed",
order_id=order_id,
customer_id=order.customer_id,
amount=order.total,
error_code=exc.code,
error_message=str(exc),
exc_info=True, # Includes full stack trace
)
raise
except Exception as exc:
# Catch-all for unexpected errors
logger.exception(
"order_processing_unexpected_error",
order_id=order_id,
error_type=type(exc).__name__,
)
raise
```
---
## 7. Performance Logging
### Timing decorator
```python
import functools
import time
from typing import Callable, TypeVar
import structlog
logger = structlog.get_logger("performance")
F = TypeVar("F", bound=Callable)
def log_duration(operation: str, slow_threshold_ms: float = 1000.0):
"""Decorator that logs the duration of a function call.
Args:
operation: A descriptive name for the operation.
slow_threshold_ms: Threshold in milliseconds above which
the log level escalates to WARNING.
"""
def decorator(func: F) -> F:
@functools.wraps(func)
async def async_wrapper(*args, **kwargs):
start = time.perf_counter()
try:
result = await func(*args, **kwargs)
return result
finally:
duration_ms = (time.perf_counter() - start) * 1000
log_fn = (
logger.warning
if duration_ms > slow_threshold_ms
else logger.debug
)
log_fn(
"operation_duration",
operation=operation,
duration_ms=round(duration_ms, 2),
slow=duration_ms > slow_threshold_ms,
)
@functools.wraps(func)
def sync_wrapper(*args, **kwargs):
start = time.perf_counter()
try:
result = func(*args, **kwargs)
return result
finally:
duration_ms = (time.perf_counter() - start) * 1000
log_fn = (
logger.warning
if duration_ms > slow_threshold_ms
else logger.debug
)
log_fn(
"operation_duration",
operation=operation,
duration_ms=round(duration_ms, 2),
slow=duration_ms > slow_threshold_ms,
)
import asyncio
if asyncio.iscoroutinefunction(func):
return async_wrapper # type: ignore
return sync_wrapper # type: ignore
return decorator
# Usage
@log_duration("fetch_user_profile", slow_threshold_ms=200)
async def get_user_profile(user_id: int) -> UserProfile:
return await user_repo.get_with_preferences(user_id)
```
### Context manager for ad-hoc timing
```python
import time
from contextlib import contextmanager
import structlog
logger = structlog.get_logger("performance")
@contextmanager
def log_timing(operation: str, **extra_fields):
"""Context manager for timing arbitrary code blocks."""
start = time.perf_counter()
yield
duration_ms = (time.perf_counter() - start) * 1000
logger.info(
"operation_duration",
operation=operation,
duration_ms=round(duration_ms, 2),
**extra_fields,
)
# Usage
async def rebuild_search_index():
with log_timing("rebuild_search_index", index="products"):
products = await product_repo.get_all()
await search_service.reindex(products)
```
### Slow query logging (SQLAlchemy)
```python
# SQLAlchemy event listener for slow queries
from sqlalchemy import event
from sqlalchemy.engine import Engine
import structlog
logger = structlog.get_logger("database")
SLOW_QUERY_THRESHOLD_MS = 500
@event.listens_for(Engine, "before_cursor_execute")
def before_cursor_execute(conn, cursor, statement, parameters, context, executemany):
conn.info.setdefault("query_start_time", []).append(time.perf_counter())
@event.listens_for(Engine, "after_cursor_execute")
def after_cursor_execute(conn, cursor, statement, parameters, context, executemany):
total_ms = (time.perf_counter() - conn.info["query_start_time"].pop()) * 1000
if total_ms > SLOW_QUERY_THRESHOLD_MS:
logger.warning(
"slow_query",
duration_ms=round(total_ms, 2),
statement=statement[:500], # Truncate long queries
parameters=str(parameters)[:200],
)
```
@@ -0,0 +1,404 @@
# Logging — TypeScript Patterns (pino)
Reference examples for the [logging skill](./SKILL.md). All patterns use [pino](https://github.com/pinojs/pino).
---
## 1. Structured Logging Setup
Configure pino once and export a factory for child loggers per module.
```typescript
// logger.ts
import pino from "pino";
export const logger = pino({
level: process.env.LOG_LEVEL ?? "info",
// Use pretty printing only in development
transport:
process.env.NODE_ENV === "development"
? { target: "pino-pretty", options: { colorize: true } }
: undefined,
// Base fields included in every log line
base: {
service: process.env.SERVICE_NAME ?? "api",
version: process.env.APP_VERSION ?? "unknown",
},
// Customize serialization
serializers: {
err: pino.stdSerializers.err,
req: pino.stdSerializers.req,
res: pino.stdSerializers.res,
},
// Redact sensitive fields (see Pattern 4)
redact: ["req.headers.authorization", "req.headers.cookie"],
});
// Create child loggers for specific modules
export function createLogger(module: string): pino.Logger {
return logger.child({ module });
}
```
```typescript
// Usage in a service
import { createLogger } from "./logger";
const log = createLogger("user-service");
export async function createUser(email: string): Promise<User> {
log.info({ email }, "creating_user");
const user = await userRepo.create({ email });
log.info({ userId: user.id, email }, "user_created");
return user;
}
```
**Output (JSON):**
```json
{"level":30,"time":1718444400123,"service":"api","module":"user-service","userId":42,"email":"alice@example.com","msg":"user_created"}
```
---
## 2. Log Levels
```typescript
import { createLogger } from "./logger";
const log = createLogger("order-service");
// DEBUG: Internal details
log.debug({ key: "user:42", hit: true, ttlRemaining: 120 }, "cache_lookup");
// INFO: Normal events
log.info({ orderId: "ORD-123", total: 99.99, items: 3 }, "order_placed");
// WARNING: Degraded state
log.warn(
{ currentRate: 450, limit: 500, windowSeconds: 60 },
"rate_limit_approaching"
);
// ERROR: Operation failure
log.error(
{ orderId: "ORD-123", provider: "stripe", errorCode: "card_declined" },
"payment_failed"
);
// FATAL: Unrecoverable
log.fatal(
{ activeConnections: 100, maxConnections: 100, waitingRequests: 47 },
"database_pool_exhausted"
);
```
---
## 3. Correlation IDs
Correlation IDs tie together all log entries from a single request. Uses Express middleware with `AsyncLocalStorage`.
```typescript
// middleware/correlation.ts
import { AsyncLocalStorage } from "node:async_hooks";
import { randomUUID } from "node:crypto";
import type { Request, Response, NextFunction } from "express";
import { logger } from "../logger";
interface RequestContext {
correlationId: string;
}
export const asyncLocalStorage = new AsyncLocalStorage<RequestContext>();
export function correlationMiddleware(
req: Request,
res: Response,
next: NextFunction
): void {
const correlationId =
(req.headers["x-correlation-id"] as string) ?? randomUUID();
res.setHeader("X-Correlation-ID", correlationId);
asyncLocalStorage.run({ correlationId }, () => {
next();
});
}
```
```typescript
// logger.ts — augment the logger to include correlation ID
import { asyncLocalStorage } from "./middleware/correlation";
export function getContextLogger(): pino.Logger {
const store = asyncLocalStorage.getStore();
if (store) {
return logger.child({ correlationId: store.correlationId });
}
return logger;
}
```
```typescript
// Usage in any module
import { getContextLogger } from "./logger";
export async function getUser(userId: number): Promise<User> {
const log = getContextLogger();
log.info({ userId }, "fetching_user");
// Output includes correlationId automatically
return await userRepo.findById(userId);
}
```
### Propagating to downstream services
When calling other microservices, forward the correlation ID:
```typescript
import { asyncLocalStorage } from "./middleware/correlation";
export async function callBillingService(userId: number): Promise<Invoice[]> {
const store = asyncLocalStorage.getStore();
const response = await fetch(
`http://billing-service/api/v1/invoices?user_id=${userId}`,
{
headers: {
"X-Correlation-ID": store?.correlationId ?? "",
},
}
);
return response.json();
}
```
---
## 4. Sensitive Data Redaction
Pino has built-in redaction support for field paths.
```typescript
import pino from "pino";
export const logger = pino({
level: "info",
redact: {
paths: [
"password",
"secret",
"token",
"apiKey",
"authorization",
"creditCard",
"req.headers.authorization",
"req.headers.cookie",
"body.password",
"body.creditCardNumber",
"*.password",
"*.secret",
],
censor: "***REDACTED***",
},
});
```
For more complex redaction (regex-based), use a custom serializer:
```typescript
// redact.ts
const CREDIT_CARD_RE = /\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b/g;
const BEARER_RE = /Bearer\s+[A-Za-z0-9\-._~+/]+=*/gi;
export function redactValue(value: unknown): unknown {
if (typeof value === "string") {
let result = value.replace(CREDIT_CARD_RE, "****-****-****-****");
result = result.replace(BEARER_RE, "Bearer ***REDACTED***");
return result;
}
if (typeof value === "object" && value !== null) {
return redactObject(value as Record<string, unknown>);
}
return value;
}
const SENSITIVE_KEYS =
/^(password|passwd|secret|token|api_?key|authorization|credit_?card|cvv|ssn)$/i;
function redactObject(obj: Record<string, unknown>): Record<string, unknown> {
const result: Record<string, unknown> = {};
for (const [key, value] of Object.entries(obj)) {
if (SENSITIVE_KEYS.test(key)) {
result[key] = "***REDACTED***";
} else {
result[key] = redactValue(value);
}
}
return result;
}
```
---
## 5. Request/Response Logging
Log every HTTP request and response with method, path, status code, duration, and body size. Uses Express middleware.
```typescript
// middleware/request-logging.ts
import type { Request, Response, NextFunction } from "express";
import { getContextLogger } from "../logger";
export function requestLogging(
req: Request,
res: Response,
next: NextFunction
): void {
const start = process.hrtime.bigint();
const log = getContextLogger();
log.info(
{
method: req.method,
path: req.originalUrl,
clientIp: req.ip,
userAgent: req.get("user-agent"),
},
"http_request_started"
);
// Hook into the response finish event
res.on("finish", () => {
const durationMs =
Number(process.hrtime.bigint() - start) / 1_000_000;
const logFn =
res.statusCode >= 500
? log.error.bind(log)
: res.statusCode >= 400
? log.warn.bind(log)
: log.info.bind(log);
logFn(
{
method: req.method,
path: req.originalUrl,
statusCode: res.statusCode,
durationMs: Math.round(durationMs * 100) / 100,
contentLength: res.get("content-length")
? parseInt(res.get("content-length")!, 10)
: undefined,
},
"http_request_completed"
);
});
next();
}
```
```typescript
// Register middleware (order matters)
app.use(correlationMiddleware);
app.use(requestLogging);
```
---
## 6. Error Logging
When logging errors, include the stack trace via pino's `err` serializer and enough context to reproduce the issue.
```typescript
import { getContextLogger } from "./logger";
const log = getContextLogger();
export async function processOrder(orderId: string): Promise<Order> {
log.info({ orderId }, "processing_order");
try {
const order = await orderRepo.findById(orderId);
if (!order) {
log.error({ orderId }, "order_not_found");
throw new OrderNotFoundError(orderId);
}
const payment = await paymentService.charge({
amount: order.total,
currency: order.currency,
customerId: order.customerId,
});
log.info(
{ orderId, paymentId: payment.id, amount: order.total },
"payment_processed"
);
return order;
} catch (err) {
if (err instanceof PaymentError) {
log.error(
{
orderId,
errorCode: err.code,
errorMessage: err.message,
err, // pino serializes Error objects with stack traces
},
"payment_failed"
);
} else {
log.error(
{
orderId,
err,
errorType: (err as Error).constructor.name,
},
"order_processing_unexpected_error"
);
}
throw err;
}
}
```
---
## 7. Performance Logging
### Timing wrapper
```typescript
import { createLogger } from "./logger";
const perfLog = createLogger("performance");
export async function withTiming<T>(
operation: string,
fn: () => Promise<T>,
slowThresholdMs = 1000
): Promise<T> {
const start = performance.now();
try {
const result = await fn();
return result;
} finally {
const durationMs = performance.now() - start;
const logFn = durationMs > slowThresholdMs ? perfLog.warn : perfLog.debug;
logFn(
{
operation,
durationMs: Math.round(durationMs * 100) / 100,
slow: durationMs > slowThresholdMs,
},
"operation_duration"
);
}
}
// Usage
const profile = await withTiming("fetch_user_profile", () =>
userRepo.getWithPreferences(userId)
);
```