A practical Python rate limiter for CLI applications

A command-line tool can overwhelm an API surprisingly quickly. A shell loop, parallel job, or retrying script may send hundreds of requests before a user notices, triggering HTTP 429 responses, temporary bans, or an account quota breach. A small Python rate limiter gives the application a predictable request schedule.

This pattern suits Linux utilities, administration scripts, data importers, and open-source command-line programs. It can enforce requests per second, preserve a modest burst capacity, and cooperate with server-provided limits such as Retry-After.

The implementation should be independent of wall-clock time, safe when commands run repeatedly, and clear about waiting. Those details matter for users working across Australian networks, where NBN performance can vary between a Sydney office, a regional Queensland site, and a remote workstation.

A good design also separates local throttling from retry handling. The limiter controls normal traffic; the retry policy responds to temporary failures. Keeping those responsibilities distinct makes the utility easier to test, package, and maintain.

Choose the right throttling model

A fixed delay is the simplest strategy: with a limit of two requests per second, sleep for half a second after every request. It is easy to understand, but it treats a one-request command and a long batch identically. It also tends to produce bursts when several processes start together.

A token bucket provides more useful behaviour. Tokens are added at a steady refill rate until the bucket reaches its capacity. Each request consumes one token. A capacity of five and a refill rate of two tokens per second allows five immediate requests, then settles at two requests per second. A leaky bucket is another option when output must be perfectly smooth, but token buckets are generally better for CLI tools.

Build a monotonic token bucket

Use time.monotonic() rather than time.time(). Wall-clock time can jump when an operating system synchronises its clock, while a monotonic clock is designed for measuring elapsed intervals. That prevents a clock adjustment from creating an accidental request burst or an unnecessarily long pause.

The limiter below is process-safe for threads because access to its state is protected by a lock. It waits only when necessary and returns immediately when a token is available.

import threading
import time

class RateLimiter:
    def __init__(self, rate, capacity=None):
        if rate <= 0:
            raise ValueError("rate must be greater than zero")
        self.rate = float(rate)
        self.capacity = float(capacity or rate)
        self.tokens = self.capacity
        self.updated = time.monotonic()
        self.lock = threading.Lock()

    def acquire(self):
        while True:
            with self.lock:
                now = time.monotonic()
                elapsed = now - self.updated
                self.tokens = min(
                    self.capacity,
                    self.tokens + elapsed * self.rate
                )
                self.updated = now

                if self.tokens >= 1:
                    self.tokens -= 1
                    return

                delay = (1 - self.tokens) / self.rate

            time.sleep(delay)

Turn limits into a reusable Python component

Expose the limiter through a small class or module rather than scattering sleep() calls throughout the command. A caller can then invoke limiter.acquire() immediately before network activity, file submission, or another operation that consumes an external quota.

Command-line options might include --requests-per-second and --burst. Use argparse to validate positive values and provide sensible defaults. For an API allowing 60 requests per minute, represent the rate as 1.0 request per second and choose a burst of three or five, depending on the service’s terms.

A limiter should not hide important delays. For an interactive utility, print an occasional status message to stderr when waiting for a noticeable period. Avoid printing for every short pause, since that makes redirected output noisy and can interfere with scripts that parse stdout.

Persist state safely between commands

A process-local bucket resets whenever the command exits. That is acceptable for a single long-running batch, but it can defeat throttling when a user launches the utility repeatedly from a shell script. Persisting the token count and timestamp in a small state file creates a limit shared by successive invocations.

Use a dedicated location such as ~/.cache/tool-name/rate-limit.json, create the directory with restrictive permissions, and write through a temporary file followed by os.replace(). The replacement prevents readers from seeing half-written JSON. Store a monotonic timestamp only for the current process, because monotonic values are not portable across boots; persist a wall-clock timestamp or, more simply, persist the last refill time using UTC and account for clock changes conservatively.

Multiple CLI processes introduce a race. On Linux, use an advisory fcntl.flock() around reading, updating, and writing the state file. Atomic replacement protects file contents, but it does not prevent two processes from both reading the same token balance and spending it. Document that the persistent mode is Unix-oriented if Windows support is outside the project’s scope.

Handle retries and server responses

A local rate limiter cannot know every server-side condition. An API may return 429 Too Many Requests, 503 Service Unavailable, or a Retry-After header even when the local schedule is correct. Parse that header when present, cap unreasonable values, and sleep before retrying. If the header contains an HTTP date rather than seconds, convert it carefully and treat invalid values as a normal backoff case.

Use exponential backoff with jitter for transient failures: for example, 1, 2, 4, and 8 seconds multiplied by a small random factor. Jitter prevents many scheduled jobs from retrying at exactly the same moment. Apply the limiter before every retry as well, so recovery traffic does not bypass the normal quota.

Set a maximum retry count and return a non-zero exit status when it is exceeded. In an automated pipeline running in Sydney or Melbourne, clear failure reporting is more useful than an apparently hung process. Include the final HTTP status and a brief reason on stderr while reserving machine-readable results for stdout.

Make the command friendly in Australian environments

Do not calculate waiting periods from local civil time. Australia spans several time zones, and daylight saving differs between states: Sydney and Melbourne change clocks while Brisbane and Perth do not. Elapsed-time scheduling with a monotonic clock avoids those differences entirely. Log timestamps in UTC or include an explicit offset.

Network conditions also vary. A command used over a congested NBN connection may already experience latency, while a server in Perth contacting an API hosted near Sydney can see a different round-trip time from a Melbourne CI runner. The limiter should regulate request starts, not assume that each response arrives within a fixed interval.

Be conservative when several users share one public IP, such as in an office, university, apartment building, or regional service. The API may apply quotas to the address or account rather than to the individual command. A configurable limit, a clear --no-wait failure mode, and plain Australian English in help text make the behaviour easier to understand during an afternoon, or “arvo”, troubleshooting session.

Test, package and tune the limiter

Test elapsed-time behaviour with a fake clock instead of relying only on real sleeps. Verify that the initial burst never exceeds capacity, that tokens refill at the configured rate, and that a blocked request eventually proceeds. Add tests for invalid rates, corrupted state files, concurrent access, Retry-After, and interruption with Ctrl-C.

Package the component with type hints, a short API document, and a licence compatible with the rest of the project. A --dry-run mode can show the calculated schedule without contacting the service. Metrics such as total waits, average delay, and retry count are useful during development, but should remain optional in a small utility.

Practical operating choices

Start with a modest burst capacity, then increase it only when the service documentation permits bursts.
Use persistent state when shell scripts or cron jobs can launch several short-lived processes.
Keep stdout stable for pipelines and send progress, waits, and diagnostics to stderr.
Honour server quotas and Retry-After headers even when the local token bucket allows a request.
Expose the rate as configuration so Australian users can tune it for local network latency and shared connections.

A well-designed limiter remains largely invisible during normal use: commands complete at a reasonable pace, API quotas are respected, and failures explain what happened without exposing internal implementation details.