Scaling Node.js on AWS: Lessons from Production

I've scaled Node.js applications on AWS for clients ranging from early-stage startups with 500 daily active users to SaaS platforms handling 200k requests per hour. The architecture patterns that let you sleep through the night are almost never what you started with — they're the result of painful incidents, runaway bills, and a few 3 AM database timeouts that taught me more than any blog post could.

This article covers everything that actually matters: instance selection, load balancing, auto-scaling, the database layer, cost control, and monitoring. No Hello World examples — only the patterns that hold up under real traffic.

"AWS gives you every tool you need to scale. The problem is it also gives you every tool you need to accidentally spend $40,000 in a month. The discipline is in knowing which knobs to touch."

Where Most Node.js Apps Break Under Load

Before talking about solutions, it's worth diagnosing where Node.js apps actually hit their ceiling. In my experience, the failure modes fall into four buckets:

• Single-process CPU saturation: Node.js is single-threaded. One CPU-intensive operation — a large JSON parse, synchronous crypto, or a tight loop — blocks the event loop for every other request during that time. The process doesn't crash; it just stops responding until it's done.
• Database connection exhaustion: Each Node.js process opens connections to Postgres or MySQL. When you add more server instances without configuring a connection pool or a connection proxy like RDS Proxy, you saturate the DB's max connection limit and every new query fails.
• Memory leaks in long-running processes: Node.js processes that run for days without restart accumulate heap objects that garbage collection never releases. RSS climbs slowly until the OOM killer arrives.
• No graceful shutdown: Auto Scaling terminates instances abruptly. If your process doesn't handle SIGTERM — drain in-flight requests, close DB connections, flush queued jobs — you get dropped requests and corrupted state on every deploy or scale-in event.

Every architectural decision I'll describe below is a direct response to one of these four failure modes.

The Architecture Stack I Use in Production

Here's the full AWS stack for a production Node.js application. Each component earns its place:

┌─────────────────────────────────────────────────────────┐
│                    INTERNET / CLIENTS                   │
└──────────────────────┬──────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────┐
│           Route 53  (DNS + Health Routing)               │
└──────────────────────┬──────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────┐
│      CloudFront CDN  (static assets, edge caching)       │
└──────────────────────┬──────────────────────────────────┘
                       │ (API requests only)
┌──────────────────────▼──────────────────────────────────┐
│   Application Load Balancer (ALB)  — HTTPS termination  │
│         Health checks · Sticky sessions (opt-in)         │
└───────┬───────────────────────────────────┬─────────────┘
        │                                   │
┌───────▼───────┐                   ┌───────▼───────┐
│  EC2 t3.medium│  ←─ Auto Scaling  │  EC2 t3.medium│  ...
│  Node.js (PM2)│     Group (ASG)   │  Node.js (PM2)│
└───────┬───────┘                   └───────┬───────┘
        └──────────────────┬────────────────────┘
                          │
          ┌───────────────┴───────────────┐
          │                               │
┌─────────▼────────┐     ┌──────────▼────────┐
│  RDS Postgres     │     │  ElastiCache Redis │
│  (Multi-AZ)       │     │  (sessions, cache) │
└──────────────────┘     └───────────────────┘
          │
┌─────────▼────────┐
│  RDS Proxy        │  ← connection pooling for DB
└──────────────────┘

The key insight in this diagram: the Node.js instances are stateless. All state — sessions, cache, persistent data — lives outside the EC2 layer. This is what makes horizontal scaling possible. You can add or remove instances at any time without data loss.

EC2 Setup: Getting the Instance Right

Choosing the Right Instance Type

For most Node.js API servers, the t3/t4g family hits the sweet spot. The t3.medium (2 vCPU, 4 GB RAM) handles 200–500 concurrent connections comfortably at moderate CPU utilization. For CPU-intensive workloads (heavy computation, PDF generation, image processing), move to the c6i compute-optimized family. Avoid over-provisioning — a t3.medium at 40% CPU is cheaper and more predictable than a m5.large at 15%.

Use the ARM-based t4g instances if your Docker images and dependencies support ARM — they're 20% cheaper than x86 equivalents for the same performance profile.

Bootstrap with User Data

Every EC2 instance should configure itself on boot without human intervention. Here's the User Data script I use as a starting point:

#!/bin/bash
# /etc/aws/user-data.sh  — runs once on first boot
set -e

# System updates
apt-get update -y && apt-get upgrade -y

# Install Node.js 20 LTS via NodeSource
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt-get install -y nodejs

# Install PM2 globally
npm install -g pm2

# Pull app code from S3 artifact or CodeDeploy
APP_DIR=/home/ubuntu/app
mkdir -p $APP_DIR
aws s3 cp s3://${DEPLOY_BUCKET}/latest/app.tar.gz /tmp/app.tar.gz
tar -xzf /tmp/app.tar.gz -C $APP_DIR

# Install dependencies (prod only)
cd $APP_DIR && npm ci --omit=dev

# Write environment variables from SSM Parameter Store
aws ssm get-parameters-by-path \
  --path "/myapp/prod/" \
  --with-decryption \
  --query "Parameters[*].[Name,Value]" \
  --output text | awk '{gsub("/myapp/prod/","",$1); print $1"="$2}' > $APP_DIR/.env

# Start app with PM2 in cluster mode (one worker per vCPU)
pm2 start ecosystem.config.js --env production
pm2 save
pm2 startup systemd -u ubuntu --hp /home/ubuntu

The ecosystem.config.js for PM2 cluster mode:

// ecosystem.config.js
module.exports = {
  apps: [{
    name: 'api',
    script: './dist/server.js',
    instances: 'max',        // one per vCPU
    exec_mode: 'cluster',
    max_memory_restart: '1G', // restart on memory leak
    env_production: {
      NODE_ENV: 'production',
      PORT: 3000
    },
    kill_timeout: 5000,       // ms to wait for graceful shutdown
    listen_timeout: 10000,
  }]
};

Load Balancing with ALB

The Application Load Balancer (ALB) is the entry point for all traffic. It handles TLS termination (keep HTTPS complexity off your Node processes), routes requests across healthy instances, and provides the health check endpoint that Auto Scaling depends on.

Critical ALB settings to get right:

• Health check path: /health — implement a dedicated lightweight endpoint in your app that checks DB connectivity and returns 200 OK in under 500ms. Don't use / — it may trigger auth middleware or expensive logic.
• Deregistration delay: Set to 30 seconds (default is 300). This is the time ALB waits after marking an instance for removal before stopping traffic. 30 seconds is enough for in-flight requests to complete for most APIs.
• Idle timeout: Match your Node.js server's keepAliveTimeout. If ALB's idle timeout (default 60s) is higher than your server's, the ALB will close connections that your app thinks are still open, causing 502 errors under load.
• Slow start duration: Give new instances 60–120 seconds to warm up before they receive full traffic weight. This prevents cold-start JIT compilation from causing latency spikes right after a scale-out event.

In your Node.js server, set keepAliveTimeout to 65 seconds — just above the ALB's 60-second idle timeout:

// server.js — graceful shutdown + keep-alive config
const server = app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});

// Must be higher than ALB idle timeout (60s)
server.keepAliveTimeout = 65000;
server.headersTimeout = 66000;

// Graceful shutdown on SIGTERM (sent by ASG on scale-in)
process.on('SIGTERM', () => {
  console.log('SIGTERM received — closing server');
  server.close(() => {
    console.log('HTTP server closed');
    pool.end().then(() => {
      console.log('DB pool drained');
      process.exit(0);
    });
  });
});

Auto Scaling Groups: The Right Configuration

An Auto Scaling Group (ASG) automatically adds EC2 instances when traffic spikes and removes them when it drops. The configuration that works in production is more nuanced than the AWS console defaults:

// CloudFormation / CDK — ASG scaling policy (simplified JSON)
{
  "AutoScalingGroup": {
    "MinSize": 2,           // never go below 2 (HA across AZs)
    "MaxSize": 12,          // cap spend
    "DesiredCapacity": 2,
    "HealthCheckType": "ELB",  // use ALB health checks, not EC2 status checks
    "HealthCheckGracePeriod": 120,  // seconds before ASG starts checking new instances
    "TargetTrackingScaling": {
      "TargetValue": 60,          // target 60% avg CPU across fleet
      "PredefinedMetricType": "ASGAverageCPUUtilization",
      "ScaleInCooldown": 300,      // 5 min before removing an instance
      "ScaleOutCooldown": 120       // 2 min before adding another
    },
    "InstanceRefreshStrategy": "Rolling",
    "MinHealthyPercentage": 80      // keep 80% healthy during deploys
  }
}

Use Target Tracking scaling policy rather than Step Scaling for most workloads. It continuously adjusts capacity to maintain your target metric (CPU, request count, or a custom CloudWatch metric). It scales out aggressively and scales in conservatively — exactly the right behavior for production.

For APIs with predictable traffic patterns (office hours in India, for example), add a Scheduled Scaling Action to pre-scale before peak load: 30 minutes before your typical morning traffic ramp, increase desired capacity to 4 instances. This prevents the 3–5 minute lag between traffic spike and new instances passing health checks from causing a degraded user experience.

Database Layer: RDS + ElastiCache

The database is almost always the first bottleneck once your Node.js layer is horizontally scaled. Every new EC2 instance opens new database connections, and Postgres's default max_connections = 100 is a hard ceiling that kills your app when you hit it.

Connection Pooling with pg-pool

Use pg with connection pooling configured per process. Keep pool size small — 5–10 connections per Node.js process is plenty. With 4 cluster workers on a 2-vCPU instance and 3 instances in the ASG, that's already 60–120 connections before RDS Proxy:

// db/pool.js
const { Pool } = require('pg');

const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
  max: 10,            // max connections per PM2 worker
  idleTimeoutMillis: 30000,
  connectionTimeoutMillis: 2000,
  ssl: { rejectUnauthorized: false },
});

// Surface pool metrics to CloudWatch
setInterval(() => {
  cloudwatch.putMetricData({
    Namespace: 'MyApp/Database',
    MetricData: [
      { MetricName: 'PoolTotal', Value: pool.totalCount },
      { MetricName: 'PoolIdle',  Value: pool.idleCount },
      { MetricName: 'PoolWaiting', Value: pool.waitingCount },
    ],
  });
}, 60000);

module.exports = pool;

For Redis (sessions, rate limiting, caching), use ioredis with ElastiCache:

// cache/redis.js
const Redis = require('ioredis');

const redis = new Redis({
  host: process.env.REDIS_HOST,   // ElastiCache primary endpoint
  port: 6379,
  maxRetriesPerRequest: 3,
  enableReadyCheck: true,
  lazyConnect: true,
  retryStrategy: (times) => Math.min(times * 100, 3000),
});

const cacheGet = async (key) => {
  const val = await redis.get(key);
  return val ? JSON.parse(val) : null;
};

const cacheSet = async (key, value, ttlSeconds = 300) => {
  await redis.setex(key, ttlSeconds, JSON.stringify(value));
};

module.exports = { redis, cacheGet, cacheSet };

Cost Control: What Actually Gets Expensive

AWS bills can surprise you fast. Here's a breakdown of where costs come from in a typical Node.js stack and what actually moves the needle:

Service	Typical Monthly Cost	How to Reduce
EC2 (On-Demand)	$120–$400	Use Reserved Instances (1yr) for baseline capacity; save 30–40%
RDS (Multi-AZ db.t3.medium)	$80–$150	Use Aurora Serverless v2 for variable workloads; stop dev DBs at night
ElastiCache (cache.t3.micro)	$20–$40	Single-node in dev; cluster mode only in prod
ALB	$25–$60	Fixed cost mostly; LCU pricing — minimize rule complexity
Data Transfer (egress)	$30–$200	CloudFront for all static + cacheable responses; biggest lever for API-heavy apps
NAT Gateway	$35–$100	Use VPC endpoints for S3 and DynamoDB; avoid routing through NAT
CloudWatch Logs	$10–$80	Set log retention to 30 days; stream to S3 for long-term archiving

The single biggest cost lever I've found: Savings Plans. A Compute Savings Plan that commits to $50/hour of EC2 usage for 1 year reduces your bill by 40–60% on compute. Combine with Spot Instances for stateless worker nodes (not the web tier) and you can cut EC2 costs by 70%.

Monitoring with CloudWatch

You're flying blind without metrics. The minimum viable monitoring setup for a Node.js + AWS stack:

• ALB — TargetResponseTime: Alert at p99 > 1000ms. This is the single best indicator of user-facing degradation.
• ALB — HTTPCode_Target_5XX_Count: Alert when > 1% of responses are 5xx over a 5-minute window.
• EC2 ASG — CPUUtilization: Informational (ASG handles scaling), but alert at sustained > 85% to flag if scaling isn't keeping up.
• RDS — DatabaseConnections: Alert when approaching max_connections. Critical — this metric predicts failure before it happens.
• RDS — FreeStorageSpace: Alert at < 20% free. RDS stops accepting writes when storage fills up — no other error, just silent writes that fail.
• ElastiCache — CacheHitRate: Alert when hit rate drops below 70% — indicates your TTL strategy may need review or your dataset has grown.
• Custom — Pool Waiting Count: If your DB pool's waiting queue grows, queries are queuing behind connection limits. Alert at > 5 waiting queries.

Set up a CloudWatch dashboard that shows all critical metrics on one screen. During incidents, tab-switching kills response time. One dashboard, one URL, shared with the entire team.

Common Mistakes That Kill Performance

After shipping and debugging Node.js + AWS stacks for several years, here are the mistakes I see most often:

• Sharing a single EC2 instance across web and background jobs. CPU-heavy jobs (report generation, email batches) will starve the web process. Use SQS + a separate worker ASG for background processing.
• No health check endpoint that actually tests the DB. A health check that just returns 200 OK unconditionally lets broken instances sit behind the ALB. Your health check should verify the DB connection responds within 500ms.
• No graceful shutdown handler. Every Auto Scaling scale-in event and every deploy will drop in-flight requests without one. Add the SIGTERM handler shown above — it's 10 lines of code that prevents hundreds of user-facing errors per deploy.
• Over-provisioned instances to compensate for inefficient code. A m5.2xlarge won't fix an N+1 query or a synchronous file read in a hot path. Profile first, scale second.
• Forgetting RDS Proxy. Without a connection proxy, each new ASG instance opens its own pool of DB connections. At scale, this exhausts Postgres's connection limit silently — queries start timing out with no obvious error log.
• No structured logging. console.log('error happened') is useless at scale. Use a structured logger (Pino, Winston) that outputs JSON. CloudWatch Logs Insights can then query your logs in seconds instead of you grep-ing across instances.