REST API Design Strategies

In 2021, a major fintech startup double-charged 10,000 customers during a Black Friday payment spike. The root cause? Their POST /charges endpoint had no idempotency protection. It processed duplicate requests when their mobile client retried after a timeout. Stripe had solved this exact problem years earlier with a single HTTP header. In the same year, Twitter’s API v1.1 was deprecated partially because it violated REST constraints so severely that caching was essentially impossible at CDN level.

REST isn’t just an architectural preference — it’s a system reliability contract. Get it right and you get horizontal scalability and CDN caching for free. Ignore it and you face double-charges, cache poisoning, and load spikes you can’t explain.

[!TIP] Interview Tip: In a System Design interview, never just say “We’ll use a REST API.” Explain why (statelessness, cacheability) and how you handle complex scenarios like pagination, versioning, and idempotency.

REST (Representational State Transfer) is not a protocol, but an architectural style for distributed hypermedia systems, defined by Roy Fielding in his 2000 PhD dissertation.

While everyone claims to build “RESTful” APIs, most are actually just “HTTP APIs”. To truly understand REST, we must look at the Richardson Maturity Model.

1. The Richardson Maturity Model

This model breaks down the principal elements of a REST approach into three steps:

The Swamp of POX

Uses HTTP as a transport protocol (usually just POST). RPC style XML/JSON.

POST /apiService
Body: <xml>getUser</xml>

Standardizing Error Responses: Problem+JSON (RFC 7807)

A common mistake in REST APIs is returning inconsistent error formats. One endpoint might return { "error": "msg" }, while another returns { "message": "error" }.

The machine-readable standard RFC 7807 (Problem+JSON) provides a consistent way to handle errors:

{
  "type": "https://example.com/probs/out-of-credit",
  "title": "You do not have enough credit.",
  "detail": "Your current balance is 30, but that costs 50.",
  "instance": "/account/12345/msgs/abc",
  "balance": 30, // Custom field
  "status": 403
}

By standardizing errors, API clients can build generic catch-all handlers for common error types (e.g., a credit checkout component that always looks for the balance field in out-of-credit types).

2. The 6 Constraints of REST

To be truly RESTful, a system must adhere to these six constraints:

Client-Server: Separation of concerns. The client handles the UI, the server handles the data.
Stateless: The server stores no session state between requests. Every request must contain all necessary information (e.g., Auth Token).
Cacheable: Responses must define themselves as cacheable or not (e.g., Cache-Control header).
Uniform Interface: Resources are identified by URLs (URIs), and manipulation is done via standard HTTP verbs.
Layered System: The client doesn’t know if it’s connected directly to the server or an intermediary (like a Load Balancer or CDN).
Code on Demand (Optional): The server can extend client functionality by sending code (e.g., JavaScript).

Deep Dive: Statelessness vs Stateful

This is often misunderstood. “Stateless” does not mean the application has no state (it has a database!). It means the Server Application Layer keeps no memory of past requests.

Stateful (Session Store):

Client logs in → Server creates SessionID: 123 in RAM → Client sends SessionID: 123.
Problem: If Server A crashes, the user is logged out. Scaling requires “Sticky Sessions”.

Stateless (Tokens):

Client logs in → Server issues a JWT (contains user_id: 1) → Client sends JWT on every request.
Benefit: Any server can handle the request. Horizontal scaling is trivial.

[!NOTE] Hardware-First Intuition: Statelessness isn’t just a “design choice”; it’s a memory optimization. If you store 1MB of session data per user in RAM, and you have 1 Million users, you need 1TB of RAM just for state. By moving state to a JWT (Client RAM) or a Shared Cache (Redis), you keep your Application Servers lightweight and replaceable.

REST Request Lifecycle Diagram

The diagram below shows how a RESTful API request flows through a system, demonstrating the 6 constraints:

RESTful API Request Lifecycle

Visualizing Statelessness, Cacheability, and Layered Architecture through a vertical flow.

CLIENT APPLICATION

Triggers Request with Auth Context

↓

⚡

CDN / EDGE CACHE

Checks Cache-Control: max-age=300

↓

⚖️

LOAD BALANCER

Layered System - Routes to healthy node

↓

🏠

STATELESS API SERVER

Statelessness - Validates JWT Tokens

↓

🗄️

DATABASE

Retrieves persistent resource state

3. HTTP Methods: Safe vs. Idempotent

Understanding these two properties is crucial for building reliable distributed systems.

Safe: The method does not change the state of the resource (Read-only).
Idempotent: Making the same request multiple times has the same effect as making it once.

Method	Usage	Safe?	Idempotent?
GET	Retrieve a resource.	✅ Yes	✅ Yes
POST	Create a new resource.	❌ No	❌ No
PUT	Update/Replace a resource.	❌ No	✅ Yes
PATCH	Partial update.	❌ No	❌ No (usually)
DELETE	Delete a resource.	❌ No	✅ Yes

3.1 Conditional Requests: E-Tags & Last-Modified

Even with safe methods like GET, we want to optimize bandwidth. If a client has already downloaded a 5MB JSON resource, they shouldn’t re-download it if it hasn’t changed.

ETag (Entity Tag): A hash of the resource content.
1. Server sends ETag: "v1-abc".
2. Client retries with If-None-Match: "v1-abc".
3. If unchanged, Server returns 304 Not Modified (Body is empty).
Last-Modified: Uses timestamps.
1. Client sends If-Modified-Since: Wed, 21 Oct 2023 07:28:00 GMT.

[!TIP] Performance Tip: E-Tags are safer than timestamps because timestamps often have 1-second precision, which is too slow for high-frequency updates.

</div>

4. Case Study: Idempotency at Stripe (PEDALS Framework)

[!IMPORTANT] The Core Challenge: How do you guarantee a payment is processed exactly once, even when networks are unreliable and clients retry?

4.1 The Scenario

You are building a payment gateway. A customer is checking out on an e-commerce site. They click “Pay $50”.

If the customer sees a spinner and clicks “Pay” again, a naive system will charge them twice.

4.2.1 The Two Generals Problem (The Intuition)

This is a classic paradox in distributed computing. Two generals need to coordinate an attack. They can only communicate via messengers who might be captured.

General A sends: “Attack at 9 AM!”
General B sends: “Received! I’ll be there.”
The Dilemma: General B doesn’t know if A received the confirmation. If A didn’t, A won’t attack, and B will die. They can send infinite confirmations, but they can never be 100% sure without a reliable channel.
API Reality: You can never be 100% sure a request was processed without an Idempotency Key.

4.3 The Solution: Idempotency Keys

Stripe solves this by requiring a unique key for state-changing operations.

Step-by-Step Flow

Client Generation: When the user clicks “Pay”, the frontend generates a UUID (e.g., 8a3f-12b9).
First Request: Client sends POST /charges with header Idempotency-Key: 8a3f-12b9.
Server Check: Stripe checks its key-value store (e.g., Redis).
- Key Not Found: This is a new request. Process the charge. Store the result (Body + Status 200) against the key.
- Key Found: This is a retry. Do not process again. Return the stored result immediately.

4.4 Trade-offs

Storage Cost: You must store response data for every key (usually with a TTL of 24 hours).
Race Conditions: If two requests with the same key arrive simultaneously, you need a Distributed Lock (like Redlock) to ensure only one processes.

4.5 Interactive Visualizer: Idempotency Simulator

Use this tool to simulate a Payment System. Adjust the Network Lag to see how timeouts affect the user experience. Observe the Network Log to see the actual headers being sent.

📱

CLIENT (MOBILE APP)

Current Status

Idle

Method Selection

SIMULATED LAG 800ms

Simulate Packet Loss (Timeout)

Stripe Account Balance

$1000

NETWORK INSPECTOR

// Waiting for traffic...

5. API Paradigms Comparison

Which style should you choose for your system?

Feature	REST	GraphQL	gRPC
Protocol	HTTP/1.1 (Text)	HTTP/1.1 (Text)	HTTP/2 (Binary)
Data Fetching	Multiple endpoints (Over-fetching)	Single endpoint (Exact fetching)	RPC methods (Strongly typed)
Performance	Good	Moderate (JSON parsing overhead)	Excellent (Protobuf is compact)
Browser Support	Universal	Universal	Requires Proxy (gRPC-Web)
Best For	Public APIs, CRUD	Complex Client Data Requirements	Internal Microservices
Schema	OpenAPI (Swagger)	GraphQL Schema (SDL)	Protocol Buffers (.proto)

[!TIP] Why gRPC for Internal Services? REST sends JSON, which is text-based and bulky. gRPC sends Protocol Buffers, which are binary and 10x smaller/faster. For internal traffic (East-West), this bandwidth saving is massive.

6. HATEOAS: The Engine of State

Hypermedia As The Engine Of Application State. This is what distinguishes a “Level 3” REST API. It means the API response contains links to valid next actions, allowing the client to navigate the API dynamically without hardcoding URLs.

Why HATEOAS?

It decouples the Client from the Server’s URL structure. If the server changes /users/1/deposit to /users/1/add-money, the client doesn’t break—it simply follows the deposit link provided in the previous response.

With HATEOAS (HAL Format):

{
  "id": 1,
  "balance": 100,
  "_links": {
  "self": { "href": "/accounts/1" },
  "deposit": { "href": "/accounts/1/deposit" },
  "withdraw": { "href": "/accounts/1/withdraw" }
  }
}

7. Content Negotiation

A truly RESTful API separates the Resource from its Representation. The client specifies what it wants using the Accept header.

Request:

GET /users/1 HTTP/1.1
Accept: application/xml

Response:

<user>
  <id>1</id>
  <name>Alice</name>
</user>

8. Pagination Strategies

When an API returns a list of items, you must paginate to protect your database.

8.1 Offset Pagination (Standard)

GET /users?page=3&limit=10

SQL: SELECT * FROM users LIMIT 10 OFFSET 20;
Pros: Simple, easy to jump to specific page.
Cons: Slow for deep pages. The DB must read 20 rows to skip them (O(N)). Not consistent if rows are inserted/deleted during browsing.

8.2 Cursor Pagination (Recommended for Feeds)

GET /users?cursor=user_id_1024&limit=10

SQL: SELECT * FROM users WHERE id > 1024 LIMIT 10;
Pros: Fast (O(1)) if indexed. Consistent even if new items are added.
Cons: Can’t jump to “Page 50”. Requires a unique, sortable field (like a Snowflake ID).

9. HTTP Status Codes Cheat Sheet

Don’t just return 200 OK for everything!

2xx: Success

200 OK: Generic success.
201 Created: Resource created successfully (use with POST).
204 No Content: Action successful, but response body is empty (use with DELETE).

3xx: Redirection

301 Moved Permanently: The resource has a new permanent URL. (Browser caches this redirect).
302 Found: Temporary redirect.
304 Not Modified: Resource hasn’t changed (Cache hit).

4xx: Client Errors

400 Bad Request: Invalid JSON, missing fields.
401 Unauthorized: Authentication failed (Missing or invalid token).
403 Forbidden: Authenticated, but not authorized (Permissions).
404 Not Found: Resource doesn’t exist.
429 Too Many Requests: Rate Limit exceeded.

5xx: Server Errors

500 Internal Server Error: Unhandled exception in code.
502 Bad Gateway: Upstream service (e.g., Database) failed.
503 Service Unavailable: Server is overloaded or down for maintenance.

10. Versioning Strategies

APIs evolve. How do you handle breaking changes?

Interactive Strategy Switcher

Select a strategy to see how the Request and Response structures change.

HTTP Request

Architectural Review

11. When NOT to use REST

REST is the default for most public APIs, but it’s not always the best choice.

Internal Microservices: Use gRPC.
- Why? REST (JSON) is text-based and slow to parse. gRPC (Protobuf) is binary, strongly typed, and supports streaming. It’s 10x faster for internal traffic.
Mobile Apps / Complex Frontends: Use GraphQL.
- Why? Mobile devices have limited battery and bandwidth. Fetching data from 5 different REST endpoints (/user, /posts, /comments) wastes resources. GraphQL fetches everything in one go.
Real-Time Data: Use WebSockets or SSE.
- Why? REST is request-response. It can’t “push” data when a stock price changes.

12. Summary

Level 2 REST (Verbs + Status Codes) is sufficient for most apps.
Use Idempotency Keys for sensitive operations (Payments) to handle retries safely.
Cursor Pagination beats Offset Pagination for large datasets.
Use standard HTTP Status Codes to communicate errors effectively.

Mnemonic for HTTP Verbs: “Get PUT to DELETE POSTing” — GET (safe, idempotent), PUT (idempotent), DELETE (idempotent), POST (neither). The two idempotent write verbs are PUT and DELETE.

Staff Engineer Tip: Use Idempotency Keys Everywhere, Not Just Payments. Any state-changing operation that can be retried needs idempotency protection: email sends (so you don’t spam users on retry), webhook deliveries, and job queue submissions. The pattern is always the same: client generates a UUID, server checks a Redis key, stores the result, and returns it on duplicate requests. Add this to your engineering standards doc and enforce it in code review.

REST API Design