Who Decides How Many Node.js Instances and Database Connections You Need?
When we learn Node.js connection pooling, we often see something simple like:
const pool = new Pool({
But this immediately raises a more important question:
Why 20?
And once your application runs multiple instances, another question appears:
If one instance has 20 database connections, what happens when I have 10 instances?
Now you're potentially talking about:
10 instances × 20 connections = 200 database connections
So who decides all of this?
Is there an algorithm?
The short answer is:
There isn't one algorithm responsible for the entire system. Different layers make different decisions.
Let's break down the system.
1. First Understand the Architecture
A typical production Node.js backend looks something like this:
Internet
│
▼
┌─────────────┐
│Load Balancer│
└──────┬──────┘
│
┌─────────┼─────────┐
▼ ▼ ▼
Node.js Node.js Node.js
Instance Instance Instance
│ │ │
Pool Pool Pool
│ │ │
└─────────┼─────────┘
▼
Database
There are several independent decisions happening here.
The load balancer decides where requests go.
The autoscaler can decide how many application instances should run.
The connection pool manages database connections inside each instance.
Engineers decide the pool configuration and database capacity based on constraints and measurements.
Let's look at each one.
2. Who Decides How Many Application Instances Run?
An application instance is simply one running copy of your Node.js application.
For example:
Node.js Application
That's one instance.
If you run three copies:
Node.js #1
Node.js #2
Node.js #3
you have three application instances.
In production, we generally don't manually watch traffic and start servers ourselves.
We use an autoscaling mechanism.
For example, Kubernetes provides the Horizontal Pod Autoscaler (HPA).
You might configure something like:
Minimum instances = 2
Maximum instances = 10
Target CPU = 60%
The autoscaler observes metrics and adjusts the number of replicas.
For example:
Low traffic
Later:
Traffic decreases
So:
The autoscaler decides the desired number of application instances based on configured policies and metrics.
3. Who Decides Which Instance Gets the Request?
That's the job of the load balancer.
Suppose we have:
Instance A
Instance B
Instance C
Incoming requests need to be distributed among them.
A simple algorithm is round robin:
Request 1 → A
Request 2 → B
Request 3 → C
Request 4 → A
Request 5 → B
Request 6 → C
Other strategies include:
least connections
weighted routing
latency-based routing
health-aware routing
The load balancer also performs health checks.
If Instance B is unhealthy:
Load Balancer
/ \
↓ ↓
Instance A Instance C
X
Instance B
unhealthy
The load balancer can stop sending traffic to it.
So:
The load balancer decides where each incoming request should go.
4. Now We Reach Connection Pooling
Once a request reaches a Node.js instance, it needs to talk to the database.
Instead of creating a new database connection for every request, the application uses a connection pool.
For example:
const pool = new Pool({
max: 15
});
The pool might contain:
Connection 1
Connection 2
Connection 3
...
Connection 15
A request borrows an available connection:
Request
│
▼
Connection Pool
│
├── Connection 1 → busy
├── Connection 2 → available
├── Connection 3 → busy
└── ...
After the query finishes, the connection becomes available again.
The important thing is:
The pool is normally created once per application instance, not once per request.
5. But Who Decides pool.max?
This is where things get interesting.
The connection pool does not magically know the perfect value.
If you write:
max: 20
you are telling the pool:
"This instance can have at most 20 connections."
But you need to determine whether 20 is actually appropriate.
And the first thing you need to know is:
How many application instances can exist?
6. The Connection Multiplication Problem
Imagine:
Maximum application instances = 10
Pool max per instance = 20
Potential database connections:
10 × 20 = 200
This is the number you need to think about.
Not: pool.max = 20
but: total possible connections = instances × pool size
This is one of the most important concepts in production connection pooling.
7. Database Capacity Is the Constraint
Suppose your database allows:
Maximum DB connections = 200
But other applications and administrative workloads need some of those connections.
You decide to reserve:
50 connections
That leaves:
200 - 50 = 150
for your Node.js application.
Suppose your application can scale to:
10 instances
Then a starting calculation is:
150 / 10 = 15
So you could configure:
const pool = new Pool({
max: 15
});
Now the theoretical maximum is:
10 instances × 15 connections = 150 connections
which fits within the application's connection budget.
This isn't a magic formula that guarantees optimal performance.
It's a capacity constraint that gives you a safe starting point.
You then validate it with load testing and production metrics.
8. What Happens If the Pool Is Full?
Suppose:
Pool size = 10
and all ten connections are busy.
Then another request arrives.
There is no immediately available connection.
Depending on the database driver/pool implementation, the request waits for a connection or eventually times out.
Conceptually:
Connection 1 → busy
If this keeps happening:
Pool exhausted
But here's an important engineering lesson:
Don't automatically solve pool exhaustion by increasing the pool size.
Why?
Because the database might already be the bottleneck.
9. The Real Root Cause Could Be Slow Queries
Suppose each database query normally takes: 50 ms
A connection is occupied for roughly 50 ms.
Now imagine a bad query causes execution time to increase to: 5 seconds
The same connection is now occupied for much longer.
So your 15 connections might become: 15 busy connections + many waiting requests
The problem isn't necessarily that your pool is too small.
The problem may be:
Slow query
The fix might be:
add the correct index
optimize the SQL
eliminate N+1 queries
reduce unnecessary database calls
improve transaction boundaries
cache frequently accessed data
10. Transactions Make Connection Usage Even More Important
Transactions need to stay on the same database connection.
This is critical.
If you acquire a connection manually, you need to make sure it gets returned to the pool.
Otherwise you can create a connection leak.
11. Connection Leaks Can Exhaust the Pool
Imagine:
Pool = 10 connections
If code repeatedly does:
const client = await pool.connect();
// use client
// forgot client.release()
connections can remain checked out.
Eventually:
Connection 1 → leaked
Connection 2 → leaked
Connection 3 → leaked
...
Connection 10 → leaked
Now:
Pool = exhausted
New requests wait or fail.
The fix is:
try {
// database work
} finally {
client.release();
}
For simple queries, using:
pool.query(...)
can be preferable because you don't need to manually manage checkout/release.
12. What Happens During a Traffic Spike?
Suppose your application normally has:
2 instances
Then traffic suddenly increases.
The autoscaler might increase the number of instances:
2
↓
4
↓
6
↓
8
But remember:
instances × pool.max
If:
pool.max = 15
then:
2 × 15 = 30
4 × 15 = 60
6 × 15 = 90
8 × 15 = 120
Your database must be capable of handling the resulting connection count.
This is why application autoscaling and database connection pooling cannot be designed independently.
13. What If the Database Is the Bottleneck?
Suppose the Node.js instances are healthy:
CPU = 40%
Memory = 50%
but:
Database CPU = 95%
Query latency = increasing
Adding more Node.js instances isn't necessarily going to help.
You could actually make things worse by generating more database traffic.
At that point, the solution might be:
Slow queries
↓
Optimize SQL
↓
Indexes
↓
Caching
↓
Read replicas
↓
Database scaling
The bottleneck determines where you scale.
14. So Is There an Algorithm That Decides Everything?
No.
Think of production infrastructure as a collection of cooperating systems.
Incoming Request
│
▼
┌─────────────┐
│Load Balancer│
└──────┬──────┘
│
decides where request goes
│
┌────────────┼────────────┐
▼ ▼ ▼
Node.js Node.js Node.js
Instance Instance Instance
│ │ │
▼ ▼ ▼
Pool Pool Pool
│ │ │
└────────────┼────────────┘
▼
Database
And alongside this:
Metrics
│
▼
Autoscaler
│
▼
Number of instances
Different components have different responsibilities.
15. Who Decides What?
| Decision | Responsible component/person |
|---|---|
| 1. Which instance receives a request? | Load balancer |
| 2. How many instances should run? | Autoscaler |
| 3. How many DB connections an instance can use? | Connection pool configuration |
4. What should pool.max be? | Engineering + capacity planning |
| 5. How many DB connections are available? | Database configuration/capacity |
| 6. Whether DB needs scaling? | Engineering + monitoring |
| 7. Whether a query needs optimization? | Developers/DB engineers |
| 8. Whether caching is needed? | Architecture/engineering |
This is the key takeaway:
Automation handles operational decisions, but engineers define the constraints and policies that automation operates within.
16. The Mental Model
When designing a Node.js backend, think in this order:
Step 1 — Measure the application
How much traffic can one instance handle while meeting your latency target?
1 instance
→ 500 requests/sec
→ p95 < 200ms
Step 2 — Determine required capacity
If you need 1,500 requests/sec:
1500 / 500
≈ 3 instances
Then add appropriate headroom and define your autoscaling range.
Step 3 — Determine the database connection budget
For example:
DB max connections = 200
Reserved = 50
Application budget = 150
Step 4 — Account for maximum instances
If maximum instances = 10:
150 / 10
= 15 connections per instance
Step 5 — Load test
Don't assume the calculation is perfect.
Measure:
CPU
Memory
Request throughput
p95/p99 latency
DB latency
Pool utilization
Connection wait time
Error rate
Step 6 — Continuously monitor
Production systems change.
Traffic changes.
Query patterns change.
Database capacity changes.
Instance sizes change.
So pool and scaling configuration should be revisited based on real measurements.
Final Takeaway
Connection pooling isn't just:
max: 20
It's part of a much bigger system.
The real relationship is:
Traffic
↓
Load Balancer
↓
Application Instances
↓
Connection Pools
↓
Database
And the critical relationship is:
Total possible DB connections = Maximum application instances × Pool max per instance
Note:The autoscaler manages application instance count.
The load balancer distributes requests.
The connection pool manages reusable database connections.
The database imposes connection and resource limits.
And engineers define the configuration, constraints, and capacity strategy.
Once you understand these relationships, connection pooling stops being a Node.js configuration detail and becomes a system-design problem.
