From Stateful to Stateless

The central architectural question is:

If yes → stateless architecture. If no → state is tied to a particular server.

                    User
                      |
                      v
                +-----------+
                |    LB     |
                +-----------+
                      |
             +--------+--------+
             |                 |
             v                 v
        +---------+       +---------+
        | Server A|       | Server B|
        +---------+       +---------+
        | Session |       | Session |
        | Cart    |       | Cart    |
        | Files   |       | Files   |
        +---------+       +---------+

User ---> Server A
User ---> Server B  X

Server A knows something Server B does not. The load balancer may therefore need sticky sessions.

                       +-------------+
                       |    Client   |
                       +-------------+
                              |
                              v
                       +-------------+
                       | LoadBalancer|
                       +-------------+
                         /     |     \
                        v      v      v
                    +-----+ +-----+ +-----+
                    | App | | App | | App |
                    |  A  | |  B  | |  C  |
                    +-----+ +-----+ +-----+
                       \       |       /
                        \      |      /
                         v     v     v
                    +-------------------+
                    | External State    |
                    | Redis / DB / S3    |
                    | Kafka              |
                    +-------------------+

Now App A, B and C are interchangeable clones. Any instance can process any request.

• Horizontal scalability
• Easier autoscaling
• Simpler failover
• Easier deployments
• Better fault isolation
• Easier container orchestration

State is information that must survive from one request/process execution to another in order to maintain application behavior.

STATE
 |
 +-- HTTP Session
 +-- Authentication
 +-- Shopping Cart
 +-- Uploaded Files
 +-- Generated Files
 +-- Local Cache
 +-- Local Database
 +-- In-memory objects
 +-- WebSocket connections
 +-- Background jobs
 +-- Locks
 +-- Temporary processing data
 +-- Rate-limit counters
 +-- Feature flags
 +-- Configuration

Not all state must disappear. The goal is to remove instance-local state from horizontally scaled application servers.

HTTP is stateless. Each request is independent:

Request 1: GET /login
Request 2: GET /account
Request 3: GET /portfolio

Applications use cookies/tokens to create continuity across requests.

Browser
   |
   | session_id
   v
App Server
   |
   +--> Find session
   +--> Identify user
   +--> Load preferences/cart/etc.

If session data lives in one server's memory, the user becomes tied to that server.

Instead of storing only a session ID, the cookie can contain small session data, usually serialized and protected.

Application data
       |
       v
Serialize
       |
       v
Encrypt / Sign
       |
       v
Cookie
       |
       v
Browser

Good for: small session data such as user ID, token, and a few preferences.

Problem: cookies are sent with requests, so large session data creates network overhead.

5 KB cookie × 100 requests = 500 KB transferred per user session

Also remember:

Use appropriate cookie protections such as Secure, HttpOnly and SameSite.

                 Browser
                    |
                    | session_id
                    v
             +-------------+
             | Load Balancer|
             +-------------+
               /    |    \
              v     v     v
           App A  App B  App C
              \     |     /
               \    |    /
                  Redis
                     |
          user/session information

The cookie carries a small session ID; Redis holds the actual session.

• Small cookie
• Large session data possible
• Any app instance can serve the user

Trade-off: Redis becomes critical infrastructure and must be highly available, monitored and capacity-planned.

User
 |
 v
Load Balancer
 |
 +----> Server A
          |
          +--- Local Session

User X ---> Server A
User X ---> Server A
User X ---> Server A

The load balancer keeps a user attached to a particular server.

Problems:

• Poor load distribution
• Harder autoscaling
• Bad failure behavior
• Harder server replacement
• Session may be lost when a server crashes

Sticky sessions preserve statefulness; they do not eliminate it.

Typical choice: small immutable information → signed/encrypted cookie/token; larger mutable session information → Redis/distributed store.

Login
  |
  v
Authentication Service
  |
  v
JWT
  |
  v
Client
  |
  | Authorization: Bearer JWT
  v
Any App Server
  |
  +--> Validate signature
  +--> Read claims
  +--> Process request

No session lookup is necessarily required.

JWT introduces trade-offs around token revocation, token size, key rotation, expiration, compromised tokens and refresh-token management.

User
 |
 v
Server A
 |
 +--> /uploads/user123.pdf

Later:

User
 |
 v
Server B
 |
 +--> File doesn't exist!

Local files make application instances stateful because Server A's filesystem is not Server B's filesystem.

             Load Balancer
                  |
          +-------+-------+
          |               |
          v               v
       Server A        Server B
       /data           /data
       file.pdf       EMPTY

Persistent user data on local disk prevents requests from being freely routed to any instance.

                 +-------------+
                 | App Server A |
                 +-------------+
                        |
                        v
                 +-------------+
                 | Object Store|
                 +-------------+
                        |
                 +------+------+
                 |             |
             document.pdf   image.jpg

Use durable object storage such as S3-compatible storage for user-generated files and large system-generated artifacts.

The application stores metadata and access information; it does not make the application instance the owner of the file.

                1. Request upload URL
Client ------------------------> API
                                  |
                                  v
                           Generate pre-signed URL
                                  |
                <-----------------+
                2. URL
Client ------------------------> Object Storage
                3. Upload

For large uploads, direct client-to-object-storage transfer reduces application-server bandwidth and improves scalability.

For processing:

Object Storage
      |
      | Event
      v
 Queue / Kafka
      |
      v
Document Processing Workers

User-generated

• Profile photos
• PDF/KYC documents
• Videos
• Customer uploads

→ Object Storage

System-generated

• Reports
• Statements
• Exports
• Invoices

Request
   |
   v
API
   |
   v
Job Queue
   |
   v
Worker
   |
   v
Generate file
   |
   v
Object Storage
   |
   v
Download URL

Avoid making an API server synchronously generate very large files.

Temporary files can also cause hidden coupling:

Server A
 |
 +-- /tmp/report.csv

Another request
 |
 v
Server B
 |
 +-- /tmp/report.csv --> NOT FOUND

Use object storage or ephemeral local storage only when losing the data is safe.

Server A: user123 -> balance = ₹10,000
Server B: user123 -> balance = ₹8,000

Caching is not inherently bad. The key question is whether the cache is the authoritative source of truth.

Database = Source of Truth
        |
        v
Redis / local cache = Derived data

Cache miss
   |
   v
Database
   |
   v
Rebuild cache

Server A:
Map<UserId, Session>
       |
       +-- Session X

Server B:
       |
       +-- No Session X

Local in-memory maps, session objects and mutable process variables break horizontal scaling when they affect correctness.

Move critical state to Redis, a database or another distributed state store.

Do not keep critical job state only in application memory.

API
 |
 v
Queue
 |
 +--> Worker A
 +--> Worker B
 +--> Worker C

Job state can live in a database, Redis, queue, or workflow engine depending on requirements.

This allows workers to be added, removed or restarted without losing the job's authoritative state.

Server A holds:
Lock(Order123)

Server A crashes
       |
       v
Who releases the lock?

Distributed systems may use database locks, Redis-based coordination or dedicated coordination systems.

However, consider whether distributed locks can be replaced with:

• Idempotency
• Optimistic concurrency
• Database constraints

These approaches can often be safer and simpler.

Client
  |
  | WebSocket
  v
Server A

Server A dies
       |
       X
Connection lost

WebSocket connections are inherently connection-stateful. At scale, events may need Pub/Sub, Kafka or another messaging mechanism.

User A connected to Server B

Server C receives event
        |
        v
      Pub/Sub
        |
        v
    Server B
        |
        v
      User A

Suppose a user is allowed 100 requests/minute. Local counters can be inconsistent:

Server A: 95 requests
Server B: 95 requests

User can effectively make ~190 requests.

Use a distributed counter/store or gateway-level distributed rate limiting.

Requests
   |
   v
 Redis / Distributed Counter
   |
   +--> request count
   +--> TTL/window
   +--> decision

Server A: featureX = ON
Server B: featureX = OFF

Different configuration across instances can create inconsistent behavior.

Configuration / Feature Flag Service
              |
       +------+------+ 
       |      |      |
       v      v      v
       A      B      C

Use centralized configuration or feature-flag mechanisms where consistency matters.

Stateful instances are difficult to terminate because users may depend on local state.

Stateless deployment:

Old: A B C
New: D E

Shift traffic to D/E

Terminate A/B/C

This enables rolling deployments, blue/green deployments, canary releases and rapid rollback.

Traffic:
100 RPS → 200 RPS → 500 RPS

Autoscaler:
3 instances → 8 instances

Because stateless instances are interchangeable, autoscaling can add capacity without needing to migrate user state between servers.

Stateless

Request
   |
   v
Server A
   X CRASH
   |
   v
Retry / new request
   |
   v
Server B
   |
   v
Success

Stateful

Request
   |
   v
Server A
   |
   +-- Local Session
   |
   X CRASH
   |
   v
Session LOST

Statelessness improves failure recovery and fault tolerance because another instance can take over.

                         INTERNET
                            |
                            v
                     +-------------+
                     | CDN / WAF   |
                     +-------------+
                            |
                            v
                     +-------------+
                     | LoadBalancer|
                     +-------------+
                            |
              +-------------+-------------+
              |             |             |
              v             v             v
          +-------+     +-------+     +-------+
          | App A |     | App B |     | App C |
          +-------+     +-------+     +-------+
              |             |             |
              +-------------+-------------+
                            |
              +-------------+-------------+
              |             |             |
              v             v             v
          +-------+      +-------+     +-------+
          | Redis |      |  DB   |     |  S3   |
          +-------+      +-------+     +-------+
                            |
                            v
                       +---------+
                       | Kafka   |
                       +---------+

The application tier becomes disposable while the state layer provides the required durability and coordination.

Statelessness has costs: network calls, latency, infrastructure complexity, operational cost and consistency concerns.

Local memory
     |
     | very fast
     v
Application

versus

Application
     |
     | network
     v
Redis

                    STATE
                      |
       +--------------+--------------+
       |              |              |
   Durable         Temporary       Derived
       |              |              |
      DB          ephemeral       Cache
       |
       v
  Must survive
  failure?

Ask these questions

1. Does this data need to survive server failure?
2. Can another instance reconstruct it?
3. Must every request see exactly the same value?
4. Can the client safely carry it?
5. Is the data large?

             Do I need state?
                    |
              +-----+-----+
              |           |
             NO          YES
              |           |
          Stateless       |
                          v
                  Must survive failure?
                       |
                 +-----+-----+
                 |           |
                NO          YES
                 |           |
              Local       External
              memory      storage
                 |           |
              Cache?      +---+---+
                          |       |
                         DB     Object
                                Store

When asked, "How would you make this application horizontally scalable?", do not immediately answer "add more servers."

Inspect these sources of hidden state:

1. Sessions
2. Authentication
3. Local files
4. Local cache
5. Local database
6. In-memory maps
7. Locks
8. Background jobs
9. WebSockets
10. Rate limits
11. Configuration
12. Temporary files

For every one ask:

This single question exposes many hidden scalability problems.

                    SERVER
                      |
        +-------------+-------------+
        |             |             |
      CPU           Memory         Disk
        |             |             |
      GOOD          CAREFUL         BAD
    ephemeral      derived       persistent
    computation     cache          state

A scalable application server should ideally behave like:

CREATE
   |
   v
USE
   |
   v
DESTROY

Critical information should not be trapped inside an application instance.

                 ┌──────────────────┐
                 │      CLIENT      │
                 └────────┬─────────┘
                          │
                          ▼
                 ┌──────────────────┐
                 │ CDN / WAF / LB   │
                 └────────┬─────────┘
                          │
             ┌────────────┼────────────┐
             │            │            │
             ▼            ▼            ▼
        ┌────────┐   ┌────────┐   ┌────────┐
        │ App A  │   │ App B  │   │ App C  │
        │STATELESS│  │STATELESS│  │STATELESS│
        └────┬───┘   └────┬───┘   └────┬───┘
             │            │            │
             └────────────┼────────────┘
                          │
           ┌──────────────┼──────────────┐
           │              │              │
           ▼              ▼              ▼
       ┌────────┐    ┌─────────┐    ┌──────────┐
       │ Redis  │    │Database │    │Object    │
       │        │    │         │    │Storage   │
       └────────┘    └─────────┘    └──────────┘
                          │
                          ▼
                    DURABLE STATE