How Frontend Developers Can Handle Millions of API Requests Without Crashing Everything
Handling millions of API requests isn’t about raw user count — it’s about preventing duplication, controlling concurrency, caching intelligently, and protecting the backend during failures. The best frontend systems combine deduplication, layered caching, backpressure, safe retries, and graceful degradation.
Uncontrolled requests are like a flash flood — they overwhelm the dam (backend). Good architecture adds filters (deduplication), reservoirs (caching), controlled gates (backpressure & concurrency limits), and emergency spillways (load shedding) to keep the system stable even under surge.
1Deduplication & Request Cancellation
Prevent multiple identical requests from firing simultaneously. Use in-flight maps and AbortController to cancel outdated work (search, filters, rapid navigation).
const inFlight = new Map();
export async function fetchOnce<T>(
key: string,
fn: () => Promise<T>
): Promise<T> {
if (inFlight.has(key)) return inFlight.get(key);
const promise = fn().finally(() => inFlight.delete(key));
inFlight.set(key, promise);
return promise;
}2Caching & Revalidation Strategy
Layer caches: React Query / SWR for memory, HTTP Cache-Control + stale-while-revalidate, CDN for edge. Combine with intelligent invalidation.
3Backpressure, Concurrency Limits & Load Shedding
Limit concurrent requests, debounce user actions, and gracefully disable non-critical features during degradation.
4Safe Retries & Idempotency
Retry only transient failures with exponential backoff + jitter. Use idempotency keys for mutations to prevent duplicates.
| Property | Naive Approach | Resilient Approach |
|---|---|---|
| Dedupe | None | In-flight + cancellation |
| Result | Retry storms & overload | Stable under load |
| Caching | Minimal | Multi-layer + SWR |
| Retries | Blind | Backoff + jitter + idempotency |
Naive Approach
Dedupe
None
Result
Retry storms & overload
Caching
Minimal
Retries
Blind
Resilient Approach
Dedupe
In-flight + cancellation
Result
Stable under load
Caching
Multi-layer + SWR
Retries
Backoff + jitter + idempotency
Common questions
- ›“How would you handle millions of API requests from the frontend?”
- ›“How do you prevent retry storms?”
- ›“What strategies do you use for caching at scale?”
- ›“How do you implement graceful degradation?”
What interviewers look for
- Holistic thinking across deduplication, caching, backpressure, and resilience
- Understanding of failure amplification
- Practical techniques (in-flight dedupe, jitter, idempotency)
- Focus on protecting backend and user experience
Short answer (60 sec)
Shape demand with deduplication, layered caching, concurrency limits, and safe retries with jitter. Use load shedding for non-critical features during outages. Observability is key.
Detailed answer (senior level)
At scale, problems come from duplication and synchronized failures. Deduplicate in-flight requests, cancel outdated ones, cache aggressively with stale-while-revalidate, limit concurrency, retry with exponential backoff + jitter, and shed non-critical load when the backend is unhealthy. Combine with idempotency for safe mutations.
- Retrying every failure without classification
- No deduplication leading to request multiplication
- Blind retries causing storms during outages
- Over-caching without proper invalidation
- No backpressure or concurrency limits
- ✓Scale failures come from duplication and synchronized retries
- ✓Deduplication and cancellation are foundational
- ✓Multi-layer caching (memory + HTTP + CDN) is high leverage
- ✓Exponential backoff + jitter prevents storms
- ✓Idempotency makes retries safe for mutations
- ✓Implement load shedding and graceful degradation
- ✓Observability turns resilience into a measurable system