Deployment Runbook - Legends of Hastinapur

Pre-Deployment Checklist

• All tests passing in CI (cargo test, cargo audit)
• Code reviewed and approved
• Database migrations tested in staging
• Release notes prepared
• Monitoring dashboards configured
• Rollback plan reviewed
• On-call engineer identified
• Maintenance window communicated (if needed)

Deployment Steps

Staging Deployment

1. Merge to staging branch

   git checkout staging
   git merge main
   git push origin staging

2. Deploy to staging server

   ssh loh-staging
   cd /opt/loh-backend
   git pull origin staging
   cargo build --release
   sudo systemctl restart loh-backend

3. Run smoke tests

   # Test authentication
   curl -X POST https://staging.loh.game/api/auth/login \
      -d '{"username":"test","password":"test123"}'
   
   # Test WebSocket connection
   wscat -c wss://staging.loh.game/ws
   
   # Check health endpoint
   curl https://staging.loh.game/health

4. Monitor for 30 minutes
   • Check Sentry for errors
   • Review server metrics (CPU, memory, connections)
   • Verify no spike in error rates

Production Deployment

1. Create release tag

   git tag -a v1.2.3 -m "Release v1.2.3"
   git push origin v1.2.3

2. Enable maintenance mode (optional)

   ssh loh-prod
   sudo touch /var/www/maintenance.flag

3. Backup database

   # Automatic backup via cron, but verify
   pg_dump loh_production > /backups/loh_prod_$(date +%Y%m%d_%H%M%S).sql

4. Deploy application

   ssh loh-prod
   cd /opt/loh-backend
   git fetch --tags
   git checkout v1.2.3
   cargo build --release

5. Run database migrations

   diesel migration run --database-url=$DATABASE_URL

6. Restart services

   sudo systemctl restart loh-backend
   sudo systemctl restart loh-game-server

7. Disable maintenance mode

   sudo rm /var/www/maintenance.flag

Post-Deployment Verification

Health Checks

# Backend health
curl https://api.loh.game/health

# WebSocket health
wscat -c wss://game.loh.game/ws

# Database connectivity
psql -h localhost -U loh_user -d loh_production -c "SELECT 1;"

Smoke Tests

1. Authentication Flow
   • Create new account
   • Login
   • Logout
2. Game Connection
   • Connect to game server
   • Send player movement
   • Verify server response
3. Core Features
   • Inventory operations
   • Combat encounter
   • Quest progress

Monitoring (First Hour)

• Metrics to watch:
  • Error rate < 0.1%
  • Response time p95 < 200ms
  • WebSocket connections stable
  • CPU usage < 70%
  • Memory usage < 80%
• Dashboards:
  • Grafana: https://metrics.loh.game
  • Sentry: https://sentry.io/loh
  • Cloudflare Analytics

Rollback Trigger Criteria

• Error rate > 5%
• Critical feature broken
• Database corruption detected
• Security vulnerability discovered

Communication

Deployment Announcement (Discord/Twitter)

🚀 Deployment in progress
Version: v1.2.3
Expected downtime: 5-10 minutes
We'll notify once complete.

Deployment Complete

✅ Deployment successful!
Version: v1.2.3 is now live
New features: [LINK TO RELEASE NOTES]
Thank you for your patience!

Common Issues & Solutions

Issue: Service won't start

# Check logs
sudo journalctl -u loh-backend -n 100

# Common causes:
# - Port already in use
sudo lsof -i :3000

# - Database connection failed
psql -h localhost -U loh_user -d loh_production

Issue: High error rate post-deployment

#fast rollback
git checkout v1.2.2
cargo build --release
sudo systemctl restart loh-backend

Emergency Contacts

• On-Call Engineer: PagerDuty
• DevOps Lead: @devops on Slack
• CTO: Direct phone (P0 incidents only)