Stateless first
Keep services free of local state and push session, cache and queue state outward. Any instance can serve any request, so scaling means adding replicas behind a load balancer, and losing a node costs nothing.
Cache in layers
Serve from the edge, then in-process, then a shared cache, and only then the database. Decide per entity how stale is acceptable, and protect hot keys from stampedes by coalescing identical requests.
Async by default
Anything a user does not wait for goes on a queue. Spikes turn into backlog instead of outages. Consumers are idempotent, retries back off, and poison messages land in a dead-letter queue to be inspected, not retried forever.
Shape data for the read
Model writes for correctness and derive read models for speed: indexes that match real queries, read replicas, denormalised views, and partitioning by tenant or time once a single node stops being enough.
Protect the system from itself
Rate limits, quotas, timeouts, circuit breakers and bulkheads. Under pressure, degrade on purpose (stale data, smaller pages, skipped enrichment) rather than failing everywhere at once.
Make it observable
Structured logs, metrics and traces share one correlation ID. Define SLOs, alert on burn rate, and keep cost per request next to latency so problems surface before users report them.
Ship small and reversible
Containers with real health checks, autoscaling on meaningful signals, canary or blue-green rollouts, feature flags, and migrations that are backward compatible so a rollback is always safe.
Boundaries follow ownership
Start as a well-modularised system and split a service only when ownership, scaling profile or release cadence demands it. Contracts between services are typed and versioned, so teams move independently.
AI is a component, not the architecture
Put models behind one gateway that owns routing, caching, fallbacks, budgets and logging. Retrieval is another data path with its own index, freshness and access rules. Agents run inside a harness: scoped tools, step and cost limits, and human checkpoints for risky actions.
Evaluate AI like any dependency
Evals are tests: golden sets, regression runs on every prompt or model change, and sampling in production. Track quality, latency and cost per task so a cheaper model or a cache hit is a measured decision, not a hope.
