Lambda Is Exhausting Your RDS Connections. Raise the Limit?
Scenario
Your API runs on Lambda functions that talk to a single RDS instance sized for normal traffic. Each invocation opens its own fresh database connection. During traffic peaks, hundreds of Lambdas run concurrently, the connection count hits the ceiling, and requests start failing with "too many connections" errors. The failures are bursty, customer-visible, and getting more frequent as usage grows.
The Quick Fix on the Table
A teammate already has the RDS parameter group open and wants to crank up max_connections. One parameter, no code changes, and the errors stop — today, at least.
Interview · Round 1
The quick fix is on the table and the room is waiting for your call. Would you sign off on it? Take a position and justify it — out loud or on paper — before revealing the analysis.
Why the Quick Fix Fails
- It moves the wall; Lambda will find it again. Lambda concurrency scales with traffic and has no idea what the database can take. Whatever the new limit is — 500, 1000 — a big enough spike opens that many connections plus one. You have scheduled the same incident for a busier day.
- Connections are not free. On Postgres each connection is a backend process with real memory cost; the practical ceiling is set by the instance's RAM, not by the parameter. Set
max_connectionshigh enough and you trade clean connection errors for memory pressure, swapping and a database that slows down for everyone, including the requests that used to succeed. - The workload itself is wasteful. A connect-per-invocation pattern pays TLS and auth handshake on every request — often more milliseconds than the query itself — and produces a storm of short-lived connections that stress exactly the code path databases are worst at. Raising the limit buys more of the wasteful thing.
- Failure gets worse, not rarer. With a low limit the overflow requests fail fast and the database stays healthy. With an inflated limit, overload manifests as database-wide degradation — slow queries, lock pileups, possible OOM — which is a far harder incident than "some requests got a connection error".
- Nothing else about the architecture improves. Cold-start connection latency, no reuse of prepared statements or warm sessions, no back-pressure from database to compute — all still absent after the parameter change.
Interview · Round 2
The interviewer nods: “Fine, the quick fix is off the table. So what exactly would you do — step by step?” Sketch your plan before revealing the approach.
The Right Approach
- Put RDS Proxy between Lambda and the database. This is the purpose-built fix: Lambdas connect to the proxy, the proxy multiplexes them onto a small, warm pool of real database connections, and idle sessions are shared. Hundreds of concurrent invocations become tens of database connections. It also holds connections through failovers, cutting failover downtime.
- Fix the connection pattern in the functions. Create the client outside the handler so warm invocations reuse it, keep pool size at 1 per execution environment, and use IAM auth or Secrets Manager via the proxy rather than embedding credentials. If you are on API Gateway + HTTP-style access patterns, consider the Data API (Aurora) where it fits.
- Cap what Lambda is allowed to do to the database. Set reserved concurrency on the DB-touching functions so total concurrency times connections-per-function stays inside the proxy pool. This is admission control: the compute layer must respect the capacity of the state layer, not discover it by crashing into it.
- Decouple spiky writes with a queue. Where the work is asynchronous, put SQS between the API and the database writers and let a smaller, steady consumer fleet drain it. The queue absorbs the spike; the database sees a flat, survivable rate.
- Tune timeouts and retries so failures don't multiply. Short connection timeouts, bounded retries with backoff and jitter — otherwise every connection failure spawns retry connections and the storm feeds itself.
- Monitor the right signals. Alarm on
DatabaseConnectionsvs pool size, proxy borrow latency, and Lambda concurrent executions — so you see saturation approaching instead of meeting it in an incident. - Only then touch max_connections. With pooling in place, if the measured steady pool genuinely needs more headroom, a modest, calculated increase (backed by instance memory math) is fine — as a tuning step, not a firefighting move.
Interview · Round 3
Final pushback: “Your plan costs more time and money than the quick fix. Convince me.” How do you defend your position under pressure?
How to Defend It
- "Raising max_connections doesn't remove the ceiling — it repaints it. Lambda concurrency is unbounded relative to the database, so any fixed number loses to a big enough spike."
- "Each Postgres connection costs real memory. Inflate the limit and the failure mode changes from 'some requests get a connection error' to 'the whole database slows down and maybe OOMs' — that's a strictly worse incident."
- "The actual defect is one-connection-per-invocation. RDS Proxy turns hundreds of Lambda connections into a small warm pool — that removes the problem instead of postponing it."
- "Reserved concurrency is our back-pressure: it makes the compute tier respect the database's capacity by design, instead of discovering it in production."
- "The proxy costs a little per hour; the parameter change costs nothing today and an outage later. I'd rather pay the small known bill."