Skip to content
← All articlesEngineering

API Rate Limiting and Throttling: A Guide for Product Teams

API rate limiting and throttling explained: algorithms, limit design, response headers and monitoring, so your API stays fast and fair as traffic grows.

Every public or partner-facing API eventually meets a client that sends too much traffic: a buggy retry loop, an over-eager integration, a scraper or simply a customer who grew faster than expected. Without limits, one noisy client can slow the service for everyone else. Rate limiting is how you keep an API predictable, fair and affordable to run.

This guide is for CTOs, founders and product managers who need to decide what to limit and why, before their engineers (in-house or outsourced) decide how.

Rate limiting vs throttling

The two terms are often used interchangeably, but they behave differently:

  • Rate limiting rejects requests above a threshold. The client gets an error and must retry later.
  • Throttling slows requests down or queues them, so work continues at a reduced pace.
  • Quotas cap total usage over a longer period, such as per day or per month, and are usually tied to a pricing plan.

Most production APIs combine all three: a short-window rate limit to protect the system, plus a quota to match the commercial plan.

Why limits matter to the business

  • Stability: a single misbehaving client cannot exhaust database connections or CPU.
  • Fairness: heavy users do not degrade the experience of everyone else.
  • Cost control: cloud bills and third-party API costs grow with traffic. Limits put a ceiling on surprises.
  • Security: limits slow brute-force logins, credential stuffing and scraping. They are one layer, not a replacement for authentication and monitoring. See our web application security checklist.
  • Monetisation: tiered limits are a simple way to separate free, standard and enterprise plans.

Common rate limiting algorithms

Algorithm How it works Good for Watch out for
Fixed window Count requests per clock window (e.g. per minute) Simplicity Bursts at window edges can double the effective rate
Sliding window Counts requests over a rolling period Smoother, fairer limits Slightly more storage and computation
Token bucket Tokens refill at a steady rate; each request spends one Allowing short bursts while capping average rate Needs sensible bucket size and refill rate
Leaky bucket Requests drain from a queue at a constant rate Smoothing traffic to a fragile backend Queued requests add latency
Concurrency limit Caps simultaneous in-flight requests Expensive endpoints such as reports or exports Needs careful timeouts so slots are released

For most business APIs, a token bucket or sliding window is a sensible default. Use a concurrency limit on heavy endpoints.

Decide what you are limiting

A single global number is rarely right. Decide the key and the scope deliberately:

  • By API key or account for authenticated APIs. This is the fairest basis for plan-based limits.
  • By user inside a tenant, so one employee’s script does not block a whole company.
  • By IP address for unauthenticated endpoints such as login, sign-up and password reset. Remember that many users can share one IP behind a corporate network or mobile carrier.
  • By endpoint: a cheap read can allow far more calls than a search, an export or a payment action.

If you run a multi-tenant product, tenant-level limits matter even more; our SaaS multi-tenancy guide covers the isolation questions that go with them.

Tell clients what is happening

Good limits are predictable. When a request is rejected, return HTTP status 429 Too Many Requests and, ideally, a Retry-After header that says how long to wait. Many APIs also expose headers showing the limit, the remaining allowance and when it resets. The exact header names vary by provider, so pick a convention and document it.

Your error body should be clear and machine-readable, with a message a developer can act on. Publish the limits in your API documentation, including how clients should back off. Recommend exponential backoff with jitter so thousands of clients do not retry at the same instant.

Where to enforce limits

  • API gateway or CDN/edge: best for coarse, high-volume protection before traffic reaches your servers.
  • Application layer: needed for business-aware rules such as per-plan, per-tenant or per-endpoint limits.
  • Shared store: if you run several application instances, counters must live in a shared fast store, otherwise each instance enforces its own limit and the real limit is multiplied.

Decide what happens if the limiter itself fails. Failing open keeps the service available but unprotected; failing closed protects the backend but may block legitimate users. The right answer depends on the endpoint, and it should be a conscious choice, not an accident.

Monitor, test and adjust

Limits are a product setting, not a one-time configuration.

  1. Measure first. Look at real traffic percentiles before choosing numbers, so you do not block normal use.
  2. Alert on 429 rates per client and per endpoint. A sudden spike may be an attack, or a customer about to churn because their integration is failing.
  3. Load test the limits and the behaviour under pressure. Our performance testing with k6 guide shows how to do this.
  4. Review regularly and communicate changes to customers before they ship.

A short checklist

  • Limits defined per account, user and endpoint where it makes sense
  • Algorithm chosen deliberately (token bucket or sliding window for most cases)
  • 429 responses with Retry-After and clear error messages
  • Limits and backoff guidance published in the API docs
  • Shared counter store for multi-instance deployments
  • Fail-open or fail-closed behaviour decided and documented
  • Dashboards and alerts for rejected requests
  • Higher, negotiated limits available for trusted partners

Getting it built well

Rate limiting touches the gateway, the application code, the data store, documentation and monitoring, so it benefits from a team that treats it as one piece of work. AkMo Technologies designs and builds APIs and web platforms for international clients; see our web development services or contact us to discuss an API that needs to scale safely.

Let’s build what’s next.

Tell us about your platform, your product idea or the systems you want to modernise. Someone from our Chennai engineering team will get back to you personally.