Docs /deployment/production

Production Deployment

Production-ready deployment strategies and best practices

Note: This is a reference doc covering generic deployment strategies (blue-green, canary, Kubernetes, etc.). For Izri's actual production setup — Railway with tag-triggered deploys via GitHub Actions — see railway.md. The sections below are kept for context if the deployment target ever changes.

📋 Overview

This guide covers deploying Izri to production environments, including pre-deployment checklists, deployment strategies, rollback procedures, and monitoring.

✅ Pre-Deployment Checklist

Code & Build

  • All tests passing
  • Code reviewed and approved
  • Dependency vulnerabilities checked (pnpm audit)
  • Production build tested locally
  • Environment variables documented
  • Database migrations ready
  • No .only or skip in tests
  • Console logs removed or gated by log level

Infrastructure

  • Database backed up
  • Secrets stored securely
  • SSL/TLS certificates valid
  • DNS records configured
  • Load balancer configured (if applicable)
  • CDN configured (if applicable)
  • Health check endpoints working

Monitoring & Observability

  • Application monitoring configured (Sentry, etc.)
  • Log aggregation set up
  • Metrics collection enabled
  • Alerting rules configured
  • Uptime monitoring active

Security

  • All secrets rotated
  • CORS configured properly
  • Rate limiting enabled
  • Security headers configured
  • Database credentials use least privilege
  • OAuth apps configured for production domains

🚀 Deployment Strategies

Strategy 1: Blue-Green Deployment

Concept: Run two identical environments (blue and green), switch traffic between them.

Benefits:

  • Zero downtime
  • Instant rollback
  • Full testing before cutover

Process:

# 1. Deploy to green environment (while blue serves traffic)
docker compose -f docker-compose.green.yml up -d --build

# 2. Run health checks on green
curl https://green.api.yourdomain.com/health

# 3. Run smoke tests
pnpm test:smoke --host=green.api.yourdomain.com

# 4. Switch load balancer from blue to green
# (This is platform-specific)

# 5. Monitor for issues
docker compose -f docker-compose.green.yml logs -f

# 6. If all good, blue becomes the next staging environment

Rollback:

# Switch load balancer back to blue
# Takes effect immediately

Strategy 2: Rolling Deployment

Concept: Update instances one at a time, maintaining availability.

Benefits:

  • Gradual rollout
  • Detect issues early
  • Resource-efficient

Process:

# Using Docker Swarm
docker service update \
  --image izri-api:v2.0.0 \
  --update-parallelism 1 \
  --update-delay 30s \
  izri-api

# Or manually with docker compose
for i in {1..3}; do
  docker compose up -d --no-deps --build api-$i
  sleep 30
  # Check health
  curl http://api-$i:4000/health
done

Rollback:

# Roll back to previous image
docker service update \
  --rollback \
  izri-api

Strategy 3: Canary Deployment

Concept: Route small percentage of traffic to new version.

Benefits:

  • Minimal risk
  • Real user feedback
  • Gradual rollout

Process:

# 1. Deploy canary version
docker compose -f docker-compose.canary.yml up -d

# 2. Configure load balancer: 95% to stable, 5% to canary
# (Platform-specific configuration)

# 3. Monitor metrics
# Compare error rates, latency, etc.

# 4. If metrics good, increase canary traffic
# 95% → 90% → 75% → 50% → 0% (all canary)

# 5. Promote canary to stable

Rollback:

# Route all traffic back to stable
# Remove canary containers
docker compose -f docker-compose.canary.yml down

🛠️ Platform-Specific Deployments

Option 1: Docker Compose (VPS)

Setup (DigitalOcean, AWS EC2, etc.):

# 1. SSH into server
ssh user@your-server.com

# 2. Clone repository
git clone https://github.com/yourorg/izri.git
cd izri

# 3. Create .env file
nano .env
# Add production environment variables

# 4. Build images
docker compose build

# 5. Start services
docker compose up -d

# 6. Run migrations
docker compose exec api pnpm db:migrate

# 7. Check health
curl http://localhost:4000/health

Update deployment:

# 1. Pull latest code
git pull origin main

# 2. Rebuild changed services
docker compose up -d --build

# 3. Check logs
docker compose logs -f

Option 2: Railway

Izri's actual production target. See railway.md for the per-service railway.toml files, env vars, the tag-triggered GitHub Actions workflow, and rollback procedures.

Option 3: Render

Configuration (render.yaml):

services:
  # API Service
  - type: web
    name: izri-api
    env: docker
    dockerfilePath: ./docker/api/Dockerfile
    dockerContext: .
    envVars:
      - key: DATABASE_URL
        fromDatabase:
          name: postgres
          property: connectionString
      - key: REDIS_URL
        fromService:
          name: redis
          type: redis
          property: connectionString
      - key: NODE_ENV
        value: production
      - key: GITHUB_CLIENT_SECRET
        sync: false  # Set manually
      - key: OPENAI_API_KEY
        sync: false
    healthCheckPath: /health

  # Web Service
  - type: web
    name: izri-web
    env: docker
    dockerfilePath: ./docker/web/Dockerfile
    dockerContext: .
    envVars:
      - key: VITE_API_URL
        value: https://izri-api.onrender.com

databases:
  - name: postgres
    databaseName: izri
    user: izri

  - name: redis
    type: redis

Deploy:

# Push to GitHub
git push origin main

# Render auto-deploys on push (if configured)
# Or manually trigger via dashboard

Option 4: Fly.io

Setup:

# 1. Install Fly CLI
curl -L https://fly.io/install.sh | sh

# 2. Login
fly auth login

# 3. Launch app
fly launch

Configuration (fly.toml):

app = "izri-api"
primary_region = "iad"

[build]
  dockerfile = "docker/api/Dockerfile"

[env]
  NODE_ENV = "production"
  PORT = "4000"

[[services]]
  internal_port = 4000
  protocol = "tcp"

  [[services.ports]]
    port = 80
    handlers = ["http"]

  [[services.ports]]
    port = 443
    handlers = ["tls", "http"]

  [[services.http_checks]]
    interval = "30s"
    timeout = "2s"
    method = "get"
    path = "/health"

[deploy]
  strategy = "rolling"

Deploy:

# Deploy
fly deploy

# Set secrets
fly secrets set \
  DATABASE_URL="postgresql://..." \
  GITHUB_CLIENT_SECRET="..." \
  OPENAI_API_KEY="sk-..."

# View logs
fly logs

# SSH into instance
fly ssh console

Option 5: Kubernetes

Deployment (k8s/deployment.yaml):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: izri-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: izri-api
  template:
    metadata:
      labels:
        app: izri-api
    spec:
      containers:
      - name: api
        image: ghcr.io/yourorg/izri-api:latest
        ports:
        - containerPort: 4000
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: izri-secrets
              key: database-url
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 4000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 4000
          initialDelaySeconds: 5
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: izri-api
spec:
  selector:
    app: izri-api
  ports:
  - port: 80
    targetPort: 4000
  type: LoadBalancer

Deploy:

# Apply configuration
kubectl apply -f k8s/

# Check status
kubectl get pods
kubectl get services

# View logs
kubectl logs -f deployment/izri-api

# Scale
kubectl scale deployment izri-api --replicas=5

📊 Database Migrations

Pre-Migration Checks

# 1. Backup database
docker compose exec postgres pg_dump -U postgres izri > backup.sql

# Or for managed databases
pg_dump $DATABASE_URL > backup-$(date +%Y%m%d-%H%M%S).sql

# 2. Verify backup
psql $DATABASE_URL < backup.sql --dry-run

Running Migrations

Development/staging first:

# Test in staging
pnpm db:migrate

# Verify application works
pnpm test:integration

Production migration:

# Option 1: Via API container
docker compose exec api pnpm db:migrate

# Option 2: Standalone script
docker run --rm \
  -e DATABASE_URL=$DATABASE_URL \
  izri-api \
  pnpm db:migrate

# Option 3: Platform-specific
railway run pnpm db:migrate
fly ssh console -C "pnpm db:migrate"

Rolling Back Migrations

# If using Drizzle with down migrations
pnpm db:rollback

# Or restore from backup
psql $DATABASE_URL < backup.sql

🔄 Rollback Procedures

Quick Rollback (Docker)

# 1. Identify previous working image
docker images izri-api

# 2. Tag and deploy
docker tag izri-api:v1.2.3 izri-api:latest
docker compose up -d --no-deps api

# 3. Monitor
docker compose logs -f api

Git-Based Rollback

# 1. Find last working commit
git log --oneline

# 2. Revert to that commit
git revert <commit-hash>

# 3. Deploy
git push origin main
# Triggers CI/CD deployment

# Or manual
docker compose up -d --build

Database Rollback

# 1. Stop application
docker compose stop api web

# 2. Restore database
psql $DATABASE_URL < backup-before-migration.sql

# 3. Deploy previous app version
docker compose up -d

# 4. Verify
curl http://localhost:4000/health

🏥 Health Checks & Monitoring

Health Check Endpoint

Implementation (src/routes/health.ts):

import { Hono } from 'hono'
import { db } from '@izri/database'
import { redis } from '../lib/redis'

const health = new Hono()

health.get('/health', async (c) => {
  const checks = {
    status: 'ok',
    timestamp: new Date().toISOString(),
    uptime: process.uptime(),
    database: 'unknown',
    redis: 'unknown',
  }

  try {
    // Check database
    await db.execute('SELECT 1')
    checks.database = 'ok'
  } catch (error) {
    checks.database = 'error'
    checks.status = 'degraded'
  }

  try {
    // Check Redis
    await redis.ping()
    checks.redis = 'ok'
  } catch (error) {
    checks.redis = 'error'
    checks.status = 'degraded'
  }

  const statusCode = checks.status === 'ok' ? 200 : 503
  return c.json(checks, statusCode)
})

export { health }

Readiness vs Liveness

Readiness: Can the service handle requests?

health.get('/ready', async (c) => {
  // More thorough checks
  const ready = 
    await checkDatabase() &&
    await checkRedis() &&
    await checkExternalAPIs()
  
  return c.json({ ready }, ready ? 200 : 503)
})

Liveness: Is the service running?

health.get('/live', async (c) => {
  // Quick check - just respond
  return c.json({ alive: true })
})

📈 Performance Optimization

1. Build Optimization

# Use buildkit for faster builds
# DOCKER_BUILDKIT=1 docker build .

# Multi-stage builds (already implemented)
# Cache dependencies separately
# Only copy what's needed to runtime

2. Resource Limits

# docker-compose.yml
services:
  api:
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 512M

3. Connection Pooling

// Database connection pool
import { drizzle } from 'drizzle-orm/node-postgres'
import pg from 'pg'

const pool = new pg.Pool({
  connectionString: env.DATABASE_URL,
  max: 20,                    // Maximum connections
  idleTimeoutMillis: 30000,   // Close idle connections
  connectionTimeoutMillis: 2000,
})

export const db = drizzle(pool)

4. Caching Strategy

import { Redis } from 'ioredis'

const redis = new Redis(env.REDIS_URL)

// Cache database queries
async function getProject(id: string) {
  const cacheKey = `project:${id}`
  
  // Check cache
  const cached = await redis.get(cacheKey)
  if (cached) return JSON.parse(cached)
  
  // Query database
  const project = await db.query.projects.findFirst({
    where: eq(projects.id, id)
  })
  
  // Store in cache
  await redis.setex(cacheKey, 300, JSON.stringify(project))
  
  return project
}

🔗 Related Documentation

Need monitoring setup? Continue with Monitoring & Logging →

Reading this with an agent? /docs/deployment/production.md serves the raw markdown.

All docs