feat: adding new skills, including testing patterns and methodologies, along with bundled resources for better usability.

2026-07-06 07:18:58 +03:00 · 2026-03-30 12:18:00 +07:00
parent 0ff5ae4082
commit 7fa9a48c6c
89 changed files with 25808 additions and 923 deletions
@@ -1,69 +1,612 @@
+---
+name: postgresql
+description: >
+  Use this skill whenever working with PostgreSQL databases, writing SQL queries, designing schemas, or optimizing database performance. Trigger on keywords like PostgreSQL, Postgres, SQL query, schema design, indexing, migrations, EXPLAIN ANALYZE, connection pooling, or any relational database operation. Also applies when debugging slow queries, setting up database tables, or working with ORMs that target PostgreSQL.
+---
+
 # PostgreSQL

-## Description
-
-PostgreSQL database patterns including queries, indexing, and optimization.
-
 ## When to Use

 - PostgreSQL database operations
 - SQL query optimization
- Schema design
+- Schema design and migrations
+- JSONB document storage within a relational model
+- Full-text search without a dedicated search engine
+- Complex analytical queries with window functions and CTEs
+
+## When NOT to Use
+
+- NoSQL-only projects where no relational database is involved
+- In-memory databases like Redis or SQLite used purely for caching or ephemeral storage
+- File-based storage scenarios that do not require a database engine

 ---

 ## Core Patterns

-### Basic Queries
+### 1. Schema Design
+
+Design tables with explicit constraints, proper types, and clear relationships.

 ```sql
-- Select with filtering
-SELECT id, name, email
-FROM users
-WHERE active = true
-ORDER BY created_at DESC
-LIMIT 20 OFFSET 0;
+-- Enums for constrained value sets
+CREATE TYPE user_role AS ENUM ('admin', 'editor', 'viewer');
+CREATE TYPE order_status AS ENUM ('pending', 'processing', 'shipped', 'delivered', 'cancelled');

-- Join
-SELECT u.*, COUNT(p.id) as post_count
+-- Composite types for reusable structures
+CREATE TYPE address AS (
+    street TEXT,
+    city   TEXT,
+    state  TEXT,
+    zip    VARCHAR(10)
+);
+
+-- Users table with constraints
+CREATE TABLE users (
+    id          BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+    email       TEXT NOT NULL UNIQUE,
+    name        TEXT NOT NULL CHECK (char_length(name) >= 1),
+    role        user_role NOT NULL DEFAULT 'viewer',
+    metadata    JSONB DEFAULT '{}',
+    created_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
+    updated_at  TIMESTAMPTZ NOT NULL DEFAULT now()
+);
+
+-- Organizations with self-referencing hierarchy
+CREATE TABLE organizations (
+    id          BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+    name        TEXT NOT NULL,
+    parent_id   BIGINT REFERENCES organizations(id) ON DELETE SET NULL,
+    created_at  TIMESTAMPTZ NOT NULL DEFAULT now()
+);
+
+-- Membership join table with composite primary key
+CREATE TABLE org_memberships (
+    user_id     BIGINT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
+    org_id      BIGINT NOT NULL REFERENCES organizations(id) ON DELETE CASCADE,
+    role        user_role NOT NULL DEFAULT 'viewer',
+    joined_at   TIMESTAMPTZ NOT NULL DEFAULT now(),
+    PRIMARY KEY (user_id, org_id)
+);
+
+-- Orders with foreign keys, check constraints, and enum status
+CREATE TABLE orders (
+    id          BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+    user_id     BIGINT NOT NULL REFERENCES users(id) ON DELETE RESTRICT,
+    status      order_status NOT NULL DEFAULT 'pending',
+    total_cents BIGINT NOT NULL CHECK (total_cents >= 0),
+    shipping    address,
+    items       JSONB NOT NULL DEFAULT '[]',
+    placed_at   TIMESTAMPTZ NOT NULL DEFAULT now()
+);
+
+-- Auto-update updated_at with a trigger
+CREATE OR REPLACE FUNCTION set_updated_at()
+RETURNS TRIGGER AS $$
+BEGIN
+    NEW.updated_at = now();
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE TRIGGER trg_users_updated_at
+    BEFORE UPDATE ON users
+    FOR EACH ROW EXECUTE FUNCTION set_updated_at();
+```
+
+**Key principles:**
+- Use `BIGINT GENERATED ALWAYS AS IDENTITY` over `SERIAL` for new projects
+- Use `TIMESTAMPTZ` (not `TIMESTAMP`) to store times with timezone awareness
+- Prefer `TEXT` over `VARCHAR(n)` unless a hard length limit is business-critical
+- Add `ON DELETE` actions on every foreign key (CASCADE, RESTRICT, or SET NULL)
+- Use `CHECK` constraints for business rules that live at the data level
+
+---
+
+### 2. Index Strategy
+
+Choose the right index type based on your query patterns.
+
+**Decision guide:**
+
+| Query Pattern | Index Type | Example |
+|---------------|-----------|---------|
+| Equality (`=`) and range (`<`, `>`, `BETWEEN`) | B-tree (default) | `WHERE created_at > '2025-01-01'` |
+| Array containment (`@>`), JSONB queries | GIN | `WHERE tags @> '{postgres}'` |
+| Full-text search (`@@`) | GIN | `WHERE to_tsvector(body) @@ query` |
+| Geometry, range overlap | GiST | `WHERE location <-> point '(40.7,-74.0)' < 0.01` |
+| Filtered subset of rows | Partial | `WHERE active = true` |
+| Index-only scans (no heap lookup) | Covering (INCLUDE) | Frequently selected columns |
+
+```sql
+-- B-tree: default, good for equality and range
+CREATE INDEX idx_orders_placed_at ON orders(placed_at DESC);
+CREATE INDEX idx_orders_user_status ON orders(user_id, status);
+
+-- GIN: arrays and JSONB containment
+CREATE INDEX idx_users_metadata ON users USING GIN (metadata);
+CREATE INDEX idx_orders_items ON orders USING GIN (items jsonb_path_ops);
+
+-- GIN: full-text search
+ALTER TABLE articles ADD COLUMN search_vector tsvector
+    GENERATED ALWAYS AS (
+        setweight(to_tsvector('english', coalesce(title, '')), 'A') ||
+        setweight(to_tsvector('english', coalesce(body, '')), 'B')
+    ) STORED;
+
+CREATE INDEX idx_articles_search ON articles USING GIN (search_vector);
+
+-- Full-text search query
+SELECT id, title, ts_rank(search_vector, query) AS rank
+FROM articles, plainto_tsquery('english', 'database optimization') AS query
+WHERE search_vector @@ query
+ORDER BY rank DESC
+LIMIT 20;
+
+-- GiST: geometry and range types
+CREATE INDEX idx_events_duration ON events USING GiST (
+    tstzrange(starts_at, ends_at)
+);
+
+-- Find overlapping events
+SELECT * FROM events
+WHERE tstzrange(starts_at, ends_at) && tstzrange('2025-06-01', '2025-06-02');
+
+-- Partial index: only index rows you actually query
+CREATE INDEX idx_orders_pending ON orders(placed_at)
+    WHERE status = 'pending';
+
+-- Covering index: avoids heap lookup for common queries
+CREATE INDEX idx_users_email_covering ON users(email)
+    INCLUDE (name, role);
+
+-- This query can now be answered entirely from the index
+SELECT name, role FROM users WHERE email = 'user@example.com';
+```
+
+**When to add an index:** Run `EXPLAIN ANALYZE` first. Add an index when you see sequential scans on large tables with selective WHERE clauses. Do not index columns with very low cardinality (e.g., a boolean on a small table) unless combined with other columns.
+
+---
+
+### 3. Query Optimization
+
+#### Reading EXPLAIN ANALYZE
+
+```sql
+EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT)
+SELECT u.name, COUNT(o.id) AS order_count
 FROM users u
-LEFT JOIN posts p ON p.user_id = u.id
-GROUP BY u.id;
+JOIN orders o ON o.user_id = u.id
+WHERE o.placed_at > now() - INTERVAL '30 days'
+GROUP BY u.id, u.name
+ORDER BY order_count DESC
+LIMIT 10;
 ```

-### Indexes
+**What to look for in the output:**
+- **Seq Scan on large tables** -- add an index or rewrite the WHERE clause
+- **Nested Loop with high row counts** -- consider a Hash Join (may need more `work_mem`)
+- **actual rows far exceeding estimated rows** -- run `ANALYZE tablename` to update statistics
+- **Buffers: shared read** large numbers -- data not cached, check `shared_buffers` sizing
+- **Sort Method: external merge** -- increase `work_mem` for this query
+
+#### Common Query Rewrites

 ```sql
-- Single column index
-CREATE INDEX idx_users_email ON users(email);
+-- BAD: correlated subquery runs once per row
+SELECT u.name,
+    (SELECT COUNT(*) FROM orders o WHERE o.user_id = u.id) AS order_count
+FROM users u;

-- Composite index
-CREATE INDEX idx_posts_user_date ON posts(user_id, created_at DESC);
+-- GOOD: single pass with JOIN + GROUP BY
+SELECT u.name, COUNT(o.id) AS order_count
+FROM users u
+LEFT JOIN orders o ON o.user_id = u.id
+GROUP BY u.id, u.name;

-- Partial index
-CREATE INDEX idx_active_users ON users(email) WHERE active = true;
+-- BAD: OR on different columns defeats index usage
+SELECT * FROM orders WHERE user_id = 5 OR status = 'pending';
+
+-- GOOD: UNION ALL lets each branch use its own index
+SELECT * FROM orders WHERE user_id = 5
+UNION ALL
+SELECT * FROM orders WHERE status = 'pending' AND user_id != 5;
+
+-- BAD: function call on indexed column prevents index use
+SELECT * FROM users WHERE LOWER(email) = 'user@example.com';
+
+-- GOOD: expression index or use citext
+CREATE INDEX idx_users_email_lower ON users(LOWER(email));
+-- or better: define email as CITEXT type
+
+-- Avoiding N+1: fetch users and their latest order in one query
+SELECT DISTINCT ON (u.id)
+    u.id, u.name, o.id AS latest_order_id, o.total_cents, o.placed_at
+FROM users u
+LEFT JOIN orders o ON o.user_id = u.id
+ORDER BY u.id, o.placed_at DESC;
 ```

-### Migrations
+---
+
+### 4. Migrations
+
+Follow the up/down pattern and plan for zero-downtime deployments.

 ```sql
-- Add column with default
-ALTER TABLE users ADD COLUMN role VARCHAR(50) DEFAULT 'user';
+-- ============================================
+-- Migration: 20250601_001_add_user_preferences
+-- ============================================

-- Add constraint
-ALTER TABLE users ADD CONSTRAINT unique_email UNIQUE (email);
+-- UP
+ALTER TABLE users ADD COLUMN preferences JSONB DEFAULT '{}';
+
+-- Create index CONCURRENTLY to avoid locking the table
+CREATE INDEX CONCURRENTLY idx_users_preferences
+    ON users USING GIN (preferences);
+
+-- DOWN
+DROP INDEX IF EXISTS idx_users_preferences;
+ALTER TABLE users DROP COLUMN IF EXISTS preferences;
 ```

+**Safe vs unsafe operations:**
+
+| Operation | Safe? | Notes |
+|-----------|-------|-------|
+| ADD COLUMN (nullable or with volatile default) | Yes | Instant in PG 11+ with non-volatile default too |
+| ADD COLUMN NOT NULL without default | No | Fails if rows exist; add nullable first, backfill, then set NOT NULL |
+| DROP COLUMN | Mostly | Quick, but ORM queries may break if they SELECT * |
+| RENAME COLUMN | Dangerous | Breaks all queries referencing old name; use a transition period |
+| ADD INDEX | Safe with CONCURRENTLY | Without CONCURRENTLY, locks writes for duration |
+| ADD CONSTRAINT (CHECK/FK) | Careful | Use NOT VALID then VALIDATE CONSTRAINT in two steps |
+| Change column type | Dangerous | Rewrites entire table; use a new column + migration instead |
+
+```sql
+-- Zero-downtime: add NOT NULL constraint safely
+-- Step 1: add column as nullable
+ALTER TABLE users ADD COLUMN phone TEXT;
+
+-- Step 2: backfill in batches
+UPDATE users SET phone = '' WHERE phone IS NULL AND id BETWEEN 1 AND 10000;
+UPDATE users SET phone = '' WHERE phone IS NULL AND id BETWEEN 10001 AND 20000;
+-- ... continue in batches
+
+-- Step 3: add constraint without full table lock
+ALTER TABLE users ADD CONSTRAINT users_phone_not_null
+    CHECK (phone IS NOT NULL) NOT VALID;
+
+-- Step 4: validate (scans table but allows concurrent writes)
+ALTER TABLE users VALIDATE CONSTRAINT users_phone_not_null;
+
+-- Step 5: optionally convert to proper NOT NULL
+ALTER TABLE users ALTER COLUMN phone SET NOT NULL;
+ALTER TABLE users DROP CONSTRAINT users_phone_not_null;
+```
+
+---
+
+### 5. JSON/JSONB
+
+Use JSONB for semi-structured data that lives alongside relational columns.
+
+**When to use JSONB:**
+- User preferences, settings, or metadata with varying keys
+- API response caching or event payloads
+- Flexible attributes that differ per row
+
+**When NOT to use JSONB:**
+- Data you regularly JOIN on or use in WHERE clauses across tables -- normalize it
+- Data that has a fixed, well-known schema -- use proper columns
+
+```sql
+-- Querying JSONB: operators
+-- ->  returns JSONB element (keeps type)
+-- ->> returns TEXT value
+-- @>  containment (left contains right)
+-- ?   key exists
+
+-- Get a nested value
+SELECT
+    metadata->>'department' AS department,
+    metadata->'settings'->>'theme' AS theme
+FROM users
+WHERE metadata @> '{"role": "admin"}';
+
+-- Check if a key exists
+SELECT * FROM users WHERE metadata ? 'avatar_url';
+
+-- Query inside JSONB arrays
+SELECT * FROM orders
+WHERE items @> '[{"sku": "WIDGET-001"}]';
+
+-- Update a nested JSONB field
+UPDATE users
+SET metadata = jsonb_set(metadata, '{settings,notifications}', '"email"')
+WHERE id = 42;
+
+-- Remove a key
+UPDATE users
+SET metadata = metadata - 'deprecated_field'
+WHERE metadata ? 'deprecated_field';
+
+-- Aggregate JSONB: expand array elements into rows
+SELECT o.id, item->>'sku' AS sku, (item->>'qty')::int AS qty
+FROM orders o, jsonb_array_elements(o.items) AS item
+WHERE o.status = 'pending';
+
+-- Index strategies for JSONB
+-- General containment queries: GIN with jsonb_ops (default)
+CREATE INDEX idx_users_metadata_gin ON users USING GIN (metadata);
+
+-- Containment-only queries (smaller, faster index): jsonb_path_ops
+CREATE INDEX idx_orders_items_path ON orders USING GIN (items jsonb_path_ops);
+
+-- Specific key lookups: expression index on extracted value
+CREATE INDEX idx_users_department ON users ((metadata->>'department'));
+```
+
+---
+
+### 6. CTEs and Window Functions
+
+#### Common Table Expressions (CTEs)
+
+```sql
+-- Readable multi-step query with CTEs
+WITH monthly_revenue AS (
+    SELECT
+        date_trunc('month', placed_at) AS month,
+        SUM(total_cents) AS revenue_cents
+    FROM orders
+    WHERE status = 'delivered'
+    GROUP BY 1
+),
+revenue_with_growth AS (
+    SELECT
+        month,
+        revenue_cents,
+        LAG(revenue_cents) OVER (ORDER BY month) AS prev_month,
+        ROUND(
+            100.0 * (revenue_cents - LAG(revenue_cents) OVER (ORDER BY month))
+            / NULLIF(LAG(revenue_cents) OVER (ORDER BY month), 0),
+            1
+        ) AS growth_pct
+    FROM monthly_revenue
+)
+SELECT * FROM revenue_with_growth ORDER BY month DESC;
+
+-- Recursive CTE: org hierarchy tree
+WITH RECURSIVE org_tree AS (
+    -- Base case: top-level orgs
+    SELECT id, name, parent_id, 0 AS depth, name::TEXT AS path
+    FROM organizations
+    WHERE parent_id IS NULL
+
+    UNION ALL
+
+    -- Recursive step
+    SELECT o.id, o.name, o.parent_id, t.depth + 1, t.path || ' > ' || o.name
+    FROM organizations o
+    JOIN org_tree t ON o.parent_id = t.id
+)
+SELECT * FROM org_tree ORDER BY path;
+```
+
+#### Window Functions
+
+```sql
+-- ROW_NUMBER: assign rank within a partition
+SELECT
+    user_id,
+    id AS order_id,
+    total_cents,
+    ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY placed_at DESC) AS rn
+FROM orders;
+
+-- Get each user's most recent order
+SELECT * FROM (
+    SELECT
+        o.*,
+        ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY placed_at DESC) AS rn
+    FROM orders o
+) sub WHERE rn = 1;
+
+-- LAG/LEAD: compare with previous/next row
+SELECT
+    placed_at::date AS order_date,
+    total_cents,
+    LAG(total_cents) OVER (ORDER BY placed_at) AS prev_order_total,
+    total_cents - LAG(total_cents) OVER (ORDER BY placed_at) AS diff
+FROM orders
+WHERE user_id = 42;
+
+-- Running total
+SELECT
+    placed_at::date AS order_date,
+    total_cents,
+    SUM(total_cents) OVER (
+        ORDER BY placed_at
+        ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
+    ) AS running_total
+FROM orders
+WHERE user_id = 42;
+
+-- NTILE: divide rows into equal buckets (e.g., quartiles)
+SELECT
+    user_id,
+    SUM(total_cents) AS lifetime_spend,
+    NTILE(4) OVER (ORDER BY SUM(total_cents) DESC) AS spend_quartile
+FROM orders
+GROUP BY user_id;
+```
+
+---
+
+### 7. Transaction Isolation
+
+PostgreSQL supports four isolation levels. The two most commonly used:
+
+| Level | Dirty Read | Non-Repeatable Read | Phantom Read | Use Case |
+|-------|-----------|-------------------|-------------|----------|
+| READ COMMITTED (default) | No | Possible | Possible | Most OLTP workloads |
+| REPEATABLE READ | No | No | No (in PG) | Reports, consistent snapshots |
+| SERIALIZABLE | No | No | No | Financial transactions, inventory |
+
+```sql
+-- Default: READ COMMITTED
+-- Each statement sees the latest committed data
+BEGIN;
+    UPDATE accounts SET balance = balance - 100 WHERE id = 1;
+    UPDATE accounts SET balance = balance + 100 WHERE id = 2;
+COMMIT;
+
+-- SERIALIZABLE: full isolation, detects write conflicts
+BEGIN ISOLATION LEVEL SERIALIZABLE;
+    -- Read current inventory
+    SELECT quantity FROM inventory WHERE sku = 'WIDGET-001';
+    -- Decrement if sufficient (PG will abort if concurrent tx conflicts)
+    UPDATE inventory SET quantity = quantity - 1 WHERE sku = 'WIDGET-001';
+COMMIT;
+-- If another SERIALIZABLE tx modified the same row, one will get:
+-- ERROR: could not serialize access due to concurrent update
+-- Your application must retry on serialization failure (SQLSTATE 40001)
+
+-- Advisory locks for application-level coordination
+SELECT pg_advisory_xact_lock(hashtext('process-user-' || '42'));
+-- Lock is held until transaction ends; no table-level contention
+```
+
+**Guidelines:**
+- Use READ COMMITTED for general CRUD operations
+- Use SERIALIZABLE when correctness requires that concurrent transactions behave as if run sequentially (e.g., balance transfers, seat reservations)
+- Always implement retry logic for serialization failures
+- Keep transactions as short as possible to reduce contention
+
+---
+
+### 8. Connection Pooling
+
+Direct PostgreSQL connections are expensive (~1-10 MB RAM each). Use a pooler.
+
+**PgBouncer configuration (pgbouncer.ini):**
+
+```ini
+[databases]
+myapp = host=127.0.0.1 port=5432 dbname=myapp
+
+[pgbouncer]
+listen_addr = 127.0.0.1
+listen_port = 6432
+auth_type = scram-sha-256
+auth_file = /etc/pgbouncer/userlist.txt
+
+; Pool mode: transaction is best for most web apps
+pool_mode = transaction
+
+; Sizing: start conservative, tune with monitoring
+default_pool_size = 20
+max_client_conn = 200
+min_pool_size = 5
+reserve_pool_size = 5
+reserve_pool_timeout = 3
+
+; Timeouts
+server_idle_timeout = 300
+client_idle_timeout = 60
+query_timeout = 30
+```
+
+**Pool sizing formula:**
+
+```
+optimal_pool_size = ((2 * cpu_cores) + effective_disk_spindles)
+```
+
+For a 4-core SSD server: `(2 * 4) + 1 = 9` connections is a good starting point. More connections does not mean more throughput -- too many causes contention.
+
+**Pool modes:**
+
+| Mode | Description | Caveats |
+|------|-------------|---------|
+| `transaction` | Connection returned after each transaction | Cannot use session-level features (LISTEN/NOTIFY, prepared statements, temp tables) |
+| `session` | Connection held for entire client session | Fewer pooling benefits; use only when session features needed |
+| `statement` | Connection returned after each statement | No multi-statement transactions; rarely used |
+
+**Application-level pooling (Python example with asyncpg):**
+
+```python
+import asyncpg
+
+pool = await asyncpg.create_pool(
+    dsn="postgresql://user:pass@localhost:6432/myapp",
+    min_size=5,
+    max_size=20,
+    max_inactive_connection_lifetime=300,
+    command_timeout=30,
+)
+
+async with pool.acquire() as conn:
+    rows = await conn.fetch("SELECT * FROM users WHERE active = true")
+```
+
+---
+
 ## Best Practices

-1. Use indexes for filtered/sorted columns
-2. Use EXPLAIN ANALYZE for slow queries
-3. Avoid SELECT * in production
-4. Use transactions for multiple operations
-5. Use connection pooling
+1. **Use parameterized queries everywhere.** Never concatenate user input into SQL strings. ORMs and query builders handle this, but verify in raw SQL contexts.
+
+2. **Run ANALYZE after bulk data changes.** The query planner relies on statistics. After large imports or deletes, run `ANALYZE tablename` to update them.
+
+3. **Prefer BIGINT for primary keys.** INTEGER (max ~2.1 billion) can be exhausted sooner than expected in high-write systems. BIGINT costs 4 extra bytes per row but avoids a painful migration later.
+
+4. **Store money as integers (cents).** Floating-point arithmetic causes rounding errors. Use `BIGINT` for cents or `NUMERIC(19,4)` if sub-cent precision is needed.
+
+5. **Add indexes for foreign keys.** PostgreSQL does not automatically index the child side of a foreign key. Without it, DELETE on the parent table triggers a sequential scan on the child.
+
+6. **Use TIMESTAMPTZ, not TIMESTAMP.** `TIMESTAMP WITHOUT TIME ZONE` silently drops timezone info. Always use `TIMESTAMPTZ` and let the application control display timezone.
+
+7. **Set statement_timeout for web requests.** Prevent runaway queries from holding connections: `SET statement_timeout = '5s';` at session start, or configure per-role in PostgreSQL.
+
+8. **Monitor with pg_stat_statements.** Enable this extension to track query performance over time. The top queries by `total_exec_time` are your optimization targets.
+
+```sql
+-- Find slowest queries
+SELECT
+    calls,
+    round(total_exec_time::numeric, 1) AS total_ms,
+    round(mean_exec_time::numeric, 1) AS mean_ms,
+    query
+FROM pg_stat_statements
+ORDER BY total_exec_time DESC
+LIMIT 10;
+```

 ## Common Pitfalls

- **N+1 queries**: Use JOINs or batch loading
- **Missing indexes**: Add indexes for WHERE/ORDER BY
- **Large transactions**: Keep transactions short
+1. **N+1 queries from ORM lazy loading.** Loading a list of users and then accessing `user.orders` in a loop generates one query per user. Use eager loading (`joinedload` in SQLAlchemy, `select_related` in Django) or batch the query with a JOIN.
+
+2. **Locking the table during migrations.** `ALTER TABLE ... ADD COLUMN NOT NULL DEFAULT 'x'` is safe in PG 11+, but `CREATE INDEX` without `CONCURRENTLY` locks writes. Always use `CREATE INDEX CONCURRENTLY` in production migrations.
+
+3. **Bloated tables from UPDATE-heavy workloads.** PostgreSQL MVCC creates dead tuples on every UPDATE. If autovacuum cannot keep up, table size and query times grow. Monitor `pg_stat_user_tables.n_dead_tup` and tune autovacuum settings for hot tables.
+
+4. **Using OFFSET for pagination on large datasets.** `OFFSET 100000` forces PG to scan and discard 100,000 rows. Use keyset pagination instead:
+
+```sql
+-- BAD: slow for deep pages
+SELECT * FROM orders ORDER BY id LIMIT 20 OFFSET 100000;
+
+-- GOOD: keyset pagination
+SELECT * FROM orders WHERE id > 100000 ORDER BY id LIMIT 20;
+```
+
+5. **Ignoring connection limits.** Each PostgreSQL connection consumes RAM. Opening hundreds of direct connections (e.g., one per serverless function invocation) will exhaust `max_connections` and crash the server. Always use PgBouncer or an application-level pool.
+
+6. **Storing large blobs in the database.** Files over a few KB should go in object storage (S3, R2). Store the URL/key in PostgreSQL. Large `bytea` or `TEXT` columns bloat the table, slow backups, and waste shared_buffers cache.
+
+## Related Skills
+
+- `databases/mongodb` - Document-based database patterns for non-relational data
+- `patterns/caching` - Caching strategies to reduce database load
+- `patterns/logging` - Logging patterns for query debugging and monitoring
@@ -0,0 +1,173 @@
+# PostgreSQL Index Decision Tree
+
+Quick reference for choosing the right index type.
+
+## Decision Tree
+
+```
+What are you querying?
+|
+-- Equality (=) or Range (<, >, BETWEEN, ORDER BY)?
+|   |
+|   +-- On a single scalar column?
+|   |   --> B-tree (default)
+|   |
+|   +-- On a timestamp/date column with append-only inserts?
+|   |   --> BRIN (much smaller than B-tree)
+|   |
+|   +-- Need the index to also return columns without table lookup?
+|       --> Covering Index (B-tree with INCLUDE)
+|
+-- Array containment (@>, &&) or JSONB queries?
+|   --> GIN
+|
+-- Full-text search (tsvector, @@)?
+|   --> GIN
+|
+-- Geometric/spatial data (points, polygons, PostGIS)?
+|   --> GiST
+|
+-- Range types (int4range, tsrange, overlaps)?
+|   --> GiST
+|
+-- Nearest-neighbor / distance queries (KNN)?
+|   --> GiST (or SP-GiST for partitioned space)
+|
+-- Only a subset of rows match your WHERE clause?
+|   --> Partial Index (any type + WHERE filter)
+|
+-- Trigram similarity (LIKE '%pattern%', pg_trgm)?
+|   --> GIN with pg_trgm  (or GiST for smaller, slower)
+|
+-- Hash equality only (= but never range)?
+    --> Hash index (rarely better than B-tree in practice)
+```
+
+## Index Type Comparison
+
+| Type | Best For | Operators | Size | Write Cost | Notes |
+|------|----------|-----------|------|------------|-------|
+| **B-tree** | Equality, range, sorting | `=  <  >  <=  >=  BETWEEN  IN  IS NULL` | Medium | Low | Default. Covers 90% of cases. |
+| **GIN** | Multi-valued data | `@>  &&  @@  ?  ?&  ?|` | Large | High (slow updates) | Best for arrays, JSONB, full-text. Use `fastupdate=on`. |
+| **GiST** | Spatial, ranges, nearest-neighbor | `<<  >>  &&  @>  <@  <->` | Medium | Medium | Lossy for some types. Supports KNN. |
+| **SP-GiST** | Partitioned search spaces | Same as GiST | Medium | Medium | Good for phone numbers, IP addresses, non-balanced trees. |
+| **BRIN** | Large sequential/append-only tables | `=  <  >  <=  >=` | Tiny | Very Low | 1000x smaller than B-tree. Only effective when physical order correlates with column values. |
+| **Hash** | Equality only | `=` | Medium | Low | WAL-logged since PG10. Rarely outperforms B-tree. |
+
+## Common Patterns
+
+### Covering Index (Index-Only Scans)
+
+Avoid heap lookups by including extra columns:
+
+```sql
+-- Query: SELECT email, name FROM users WHERE email = ?
+CREATE INDEX idx_users_email_covering
+    ON users (email) INCLUDE (name);
+```
+
+### Partial Index (Filtered)
+
+Index only the rows you actually query:
+
+```sql
+-- Only index active orders (skip 95% of rows)
+CREATE INDEX idx_orders_active
+    ON orders (created_at)
+    WHERE status = 'active';
+```
+
+### Composite Index (Multi-Column)
+
+Column order matters -- put equality columns first, range columns last:
+
+```sql
+-- Query: WHERE tenant_id = ? AND created_at > ?
+CREATE INDEX idx_events_tenant_date
+    ON events (tenant_id, created_at);
+```
+
+### Expression Index
+
+Index a computed value:
+
+```sql
+CREATE INDEX idx_users_lower_email
+    ON users (lower(email));
+```
+
+### GIN for JSONB
+
+```sql
+-- Index all keys and values in a JSONB column
+CREATE INDEX idx_metadata_gin
+    ON products USING gin (metadata jsonb_path_ops);
+
+-- Supports: metadata @> '{"color": "red"}'
+```
+
+### GiST for Range Overlap
+
+```sql
+CREATE INDEX idx_reservations_during
+    ON reservations USING gist (during);
+
+-- Supports: WHERE during && '[2025-01-01, 2025-01-31]'::daterange
+```
+
+### BRIN for Time-Series
+
+```sql
+-- Table has millions of rows inserted in timestamp order
+CREATE INDEX idx_logs_ts_brin
+    ON logs USING brin (created_at)
+    WITH (pages_per_range = 32);
+```
+
+## Sizing Rules of Thumb
+
+| Table Rows | B-tree Size | BRIN Size | GIN Size |
+|------------|-------------|-----------|----------|
+| 1M | ~20 MB | ~50 KB | ~30 MB |
+| 10M | ~200 MB | ~500 KB | ~300 MB |
+| 100M | ~2 GB | ~5 MB | ~3 GB |
+
+## Diagnostic Queries
+
+```sql
+-- Check if an index is being used
+EXPLAIN (ANALYZE, BUFFERS) SELECT ...;
+
+-- Find unused indexes
+SELECT indexrelname, idx_scan
+FROM pg_stat_user_indexes
+WHERE idx_scan = 0
+ORDER BY pg_relation_size(indexrelid) DESC;
+
+-- Check index size
+SELECT pg_size_pretty(pg_relation_size('idx_name'));
+
+-- Index bloat estimate
+SELECT * FROM pgstatindex('idx_name');
+```
+
+## Anti-Patterns
+
+| Mistake | Why It Hurts |
+|---------|-------------|
+| Indexing every column | Slows writes, wastes disk, confuses planner |
+| Wrong column order in composite | Index cannot be used for the query |
+| GIN on tiny tables | Overhead exceeds benefit |
+| B-tree on low-cardinality columns | Planner prefers seq scan anyway |
+| Missing `CONCURRENTLY` on production | Locks the table during index build |
+| Forgetting `ANALYZE` after bulk load | Planner uses stale statistics |
+
+## Safe Index Creation
+
+```sql
+-- Non-blocking index creation (no table lock)
+CREATE INDEX CONCURRENTLY idx_name ON table (column);
+
+-- Always run ANALYZE after bulk operations
+ANALYZE table;
+```
@@ -0,0 +1,143 @@
+-- =============================================================================
+-- Migration: [DESCRIPTION]
+-- Created:   [DATE]
+-- Author:    [AUTHOR]
+-- Ticket:    [TICKET-ID]
+-- =============================================================================
+--
+-- SAFETY CHECKLIST (review before running):
+--   [ ] Tested on staging with production-size data
+--   [ ] Backward compatible with current application code
+--   [ ] No exclusive locks on large tables during peak hours
+--   [ ] Rollback (DOWN) section tested independently
+--   [ ] Estimated run time: ___
+--   [ ] Estimated lock duration: ___
+--
+
+-- ============================================================
+-- UP MIGRATION
+-- ============================================================
+
+BEGIN;
+
+-- Set a statement timeout to prevent long-running locks.
+-- Adjust as needed; remove for data-only migrations.
+SET LOCAL lock_timeout = '5s';
+SET LOCAL statement_timeout = '30s';
+
+-- ------------------------------------
+-- 1. Schema changes
+-- ------------------------------------
+
+-- Add new table
+-- CREATE TABLE IF NOT EXISTS example (
+--     id          bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+--     name        text NOT NULL,
+--     created_at  timestamptz NOT NULL DEFAULT now(),
+--     updated_at  timestamptz NOT NULL DEFAULT now()
+-- );
+
+-- Add column (safe: does not rewrite table)
+-- ALTER TABLE example ADD COLUMN IF NOT EXISTS description text;
+
+-- Add column with default (PG 11+: does not rewrite table)
+-- ALTER TABLE example ADD COLUMN IF NOT EXISTS is_active boolean NOT NULL DEFAULT true;
+
+-- Rename column (safe: metadata-only change)
+-- ALTER TABLE example RENAME COLUMN old_name TO new_name;
+
+-- ------------------------------------
+-- 2. Constraints
+-- ------------------------------------
+
+-- Add NOT NULL (requires all existing rows to satisfy it)
+-- ALTER TABLE example ALTER COLUMN name SET NOT NULL;
+
+-- Add check constraint (NOT VALID avoids full table scan, then VALIDATE separately)
+-- ALTER TABLE example ADD CONSTRAINT chk_example_name CHECK (name <> '') NOT VALID;
+-- ALTER TABLE example VALIDATE CONSTRAINT chk_example_name;
+
+-- Add foreign key (NOT VALID + VALIDATE pattern to avoid long locks)
+-- ALTER TABLE example ADD CONSTRAINT fk_example_parent
+--     FOREIGN KEY (parent_id) REFERENCES parent(id) NOT VALID;
+-- ALTER TABLE example VALIDATE CONSTRAINT fk_example_parent;
+
+-- ------------------------------------
+-- 3. Indexes (use CONCURRENTLY outside transaction)
+-- ------------------------------------
+
+-- NOTE: CREATE INDEX CONCURRENTLY cannot run inside a transaction.
+-- Run these statements separately after committing the transaction above.
+--
+-- CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_example_name
+--     ON example (name);
+--
+-- CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_example_created_at
+--     ON example USING brin (created_at);
+
+-- ------------------------------------
+-- 4. Data migration
+-- ------------------------------------
+
+-- Backfill in batches to avoid long transactions:
+-- UPDATE example SET description = 'default' WHERE description IS NULL;
+--
+-- For large tables, batch with:
+-- DO $$
+-- DECLARE
+--     batch_size int := 10000;
+--     rows_updated int;
+-- BEGIN
+--     LOOP
+--         UPDATE example
+--         SET description = 'default'
+--         WHERE id IN (
+--             SELECT id FROM example
+--             WHERE description IS NULL
+--             LIMIT batch_size
+--             FOR UPDATE SKIP LOCKED
+--         );
+--         GET DIAGNOSTICS rows_updated = ROW_COUNT;
+--         EXIT WHEN rows_updated = 0;
+--         RAISE NOTICE 'Updated % rows', rows_updated;
+--         COMMIT;
+--     END LOOP;
+-- END $$;
+
+-- ------------------------------------
+-- 5. Permissions
+-- ------------------------------------
+
+-- GRANT SELECT, INSERT, UPDATE ON example TO app_role;
+-- GRANT USAGE ON SEQUENCE example_id_seq TO app_role;
+
+COMMIT;
+
+
+-- ============================================================
+-- DOWN MIGRATION (rollback)
+-- ============================================================
+-- Run this section to undo the UP migration.
+-- Test this independently before deploying the UP migration.
+
+-- BEGIN;
+--
+-- -- Reverse data migration
+-- -- UPDATE example SET description = NULL;
+--
+-- -- Drop constraints
+-- -- ALTER TABLE example DROP CONSTRAINT IF EXISTS chk_example_name;
+-- -- ALTER TABLE example DROP CONSTRAINT IF EXISTS fk_example_parent;
+--
+-- -- Drop columns
+-- -- ALTER TABLE example DROP COLUMN IF EXISTS description;
+-- -- ALTER TABLE example DROP COLUMN IF EXISTS is_active;
+--
+-- -- Drop tables
+-- -- DROP TABLE IF EXISTS example;
+--
+-- COMMIT;
+--
+-- -- Drop indexes (outside transaction)
+-- -- DROP INDEX CONCURRENTLY IF EXISTS idx_example_name;
+-- -- DROP INDEX CONCURRENTLY IF EXISTS idx_example_created_at;