How would you design a rate limiter that allows at most N requests per second?

There are a few common ways to implement this. One approach pairs a Semaphore initialized with N permits with a ScheduledExecutor that refills those N permits back once every second, and each incoming request calls acquire() before it's allowed to proceed. A second approach is a token bucket built on an AtomicLong that tracks both the current token count and the time of the last refill: on each incoming request, you compute how much time has elapsed since the last refill, add tokens to the bucket proportionally to that elapsed time, and if at least one token is available you decrement it and allow the request through, otherwise you reject it, with the whole read-modify-write sequence implemented as a CAS retry loop to keep it thread-safe. A third, simpler option in a single JVM is to reach for an existing library, such as Guava's RateLimiter.create(N), whose acquire() call blocks the caller until a permit becomes available. In a real distributed production system running across many instances, you would typically move this logic to Redis with an atomic Lua script, so the rate limit is enforced globally rather than separately per instance.