bash# Capture current state# 1. p50/p99/p999 per endpoint from APM# 2. DB: avg queries per request per endpoint# 3. Cache hit rate# 4. Connection pool: active/idle/waiting# 5. CPU and memory utilization# Document in your incident/tuning notes:echo "Baseline: $(date)"echo "p99: [from APM]"echo "DB queries/req: [from APM]"echo "Cache hit rate: $(redis-cli INFO stats | grep keyspace)"
Work through this checklist in order:
sql-- Top queries by total timeSELECT query, calls, mean_exec_time, total_exec_timeFROM pg_stat_statementsORDER BY total_exec_time DESCLIMIT 20;-- Queries with high call count (N+1 suspects)SELECT query, callsFROM pg_stat_statementsWHERE calls > 10000ORDER BY calls DESC;
sqlSELECT schemaname, tablename, seq_scan, seq_tup_read,idx_scan, idx_tup_fetchFROM pg_stat_user_tablesWHERE seq_scan > idx_scanAND n_live_tup > 10000ORDER BY seq_scan DESC;
# Check pool metrics in Prometheus/Datadog:db_pool_connections_total{state="active"} / max_pool_sizedb_pool_wait_duration_seconds (p99)
bashredis-cli INFO stats | grep -E "keyspace_hits|keyspace_misses"# Calculate: hit_rate = hits / (hits + misses)
| Bottleneck Found | Fix | Expected Improvement |
|---|---|---|
| N+1 queries | Add eager loading / IN batch | 10-100x query count reduction |
| Missing index | CREATE INDEX CONCURRENTLY | 100-1000x for point lookups |
| Pool undersized | Increase max_connections | p99 wait drops proportionally |
| Low cache hit rate | Increase TTL or cache size | Direct improvement per equation |
| CPU-bound work blocking async | Move to thread/process pool | Unblocks event loop |
After applying fix:
../../bsps/07-core-backend-engineering/01-n-plus-one-query-problem.md../../bsps/07-core-backend-engineering/02-connection-pooling.md../../bsps/07-core-backend-engineering/03-caching-strategy.md../../bsps/09-performance-engineering/01-profiling-and-benchmarking.mdin this section