Skip to main content

Deployment Guide

Comprehensive deployment guide for the FeelyFeely CDN service covering various deployment scenarios, from development to production.

Deployment Overview

The FF-CDN supports multiple deployment strategies:

  • Development: Local development server
  • Staging: Pre-production testing environment
  • Production: High-availability production deployment
  • Container: Docker-based deployment
  • Platform-as-a-Service: Heroku, DigitalOcean App Platform

Prerequisites

System Requirements

Minimum Requirements

  • CPU: 2 cores
  • RAM: 4GB
  • Storage: 20GB SSD
  • Network: 100Mbps bandwidth
  • CPU: 4+ cores
  • RAM: 8GB+
  • Storage: 50GB+ SSD
  • Network: 1Gbps bandwidth
  • Load Balancer: For high availability

Dependencies

System Packages

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y \
python3.11 python3.11-dev python3.11-venv \
build-essential \
curl \
python3-opencv \
imagemagick \
libsm6 libxext6 \
libpq-dev \
nginx

# CentOS/RHEL
sudo yum install -y \
python3.11 python3.11-devel \
gcc gcc-c++ make \
curl \
opencv-python \
ImageMagick \
libSM libXext \
postgresql-devel \
nginx

Development Deployment

Local Development Setup

  1. Clone and Setup

    git clone https://github.com/FeelyFeely/ff-cdn.git
    cd ff-cdn

    # Create virtual environment
    python3.11 -m venv venv
    source venv/bin/activate

    # Install dependencies
    pip install -r requirements.txt
  2. Environment Configuration

    # Copy example environment file
    cp .env.example .env

    # Edit with development settings
    nano .env
  3. Run Development Server

    # Development mode with auto-reload
    python server.py

    # The server will be available at http://localhost:5065

Development Environment Variables

# Development-specific settings
ENVIRONMENT=development
FLASK_ENV=development
DEBUG=True
SSL_VERIFY=False

# Database (use local or development instances)
MYSQL_HOST=localhost
MYSQL_PORT=3306
MONGODB_MAIN_PROD=mongodb://localhost:27017/gateway_dev

# Reduced timeouts for faster development
REQUEST_TIMEOUT=10
UPLOAD_TIMEOUT=30

Staging Deployment

Staging Server Setup

  1. Server Preparation

    # Create application user
    sudo useradd -m -s /bin/bash ff-cdn
    sudo mkdir -p /opt/ff-cdn
    sudo chown ff-cdn:ff-cdn /opt/ff-cdn

    # Switch to application user
    sudo su - ff-cdn
  2. Application Deployment

    cd /opt/ff-cdn

    # Clone repository
    git clone https://github.com/FeelyFeely/ff-cdn.git .

    # Setup virtual environment
    python3.11 -m venv venv
    source venv/bin/activate

    # Install dependencies
    pip install -r requirements.txt
  3. Environment Configuration

    # Create staging environment file
    cat > .env << EOF
    ENVIRONMENT=staging
    FLASK_ENV=staging
    DEBUG=True

    # Staging database connections
    MYSQL_HOST=staging-db.example.com
    MONGODB_MAIN_PROD=mongodb://staging-mongo.example.com/gateway_staging

    # Staging storage
    DO_USER_ASSET_SPACES_BUCKET_NAME=ff.user.assets.staging

    # Other staging-specific configurations...
    EOF
  4. Run with Gunicorn

    # Install Gunicorn (already in requirements.txt)
    source venv/bin/activate

    # Start Gunicorn server
    gunicorn --config gunicorn_config.py wsgi:app

Staging Systemd Service

Create systemd service file:

sudo tee /etc/systemd/system/ff-cdn-staging.service << EOF
[Unit]
Description=FeelyFeely CDN Staging
After=network.target

[Service]
Type=notify
User=ff-cdn
Group=ff-cdn
WorkingDirectory=/opt/ff-cdn
Environment=PATH=/opt/ff-cdn/venv/bin
ExecStart=/opt/ff-cdn/venv/bin/gunicorn --config gunicorn_config.py wsgi:app
ExecReload=/bin/kill -s HUP \$MAINPID
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target
EOF

# Enable and start service
sudo systemctl daemon-reload
sudo systemctl enable ff-cdn-staging
sudo systemctl start ff-cdn-staging

Production Deployment

Production Server Architecture

graph TB
subgraph "Load Balancer"
LB[Nginx/HAProxy]
end

subgraph "Application Servers"
APP1[FF-CDN Server 1]
APP2[FF-CDN Server 2]
APP3[FF-CDN Server 3]
end

subgraph "Database Layer"
MYS[MySQL Primary]
MYR[MySQL Replica]
MON[MongoDB Cluster]
end

subgraph "Storage"
DOS[DigitalOcean Spaces]
UPC[Uploadcare CDN]
end

LB --> APP1
LB --> APP2
LB --> APP3

APP1 --> MYS
APP2 --> MYS
APP3 --> MYS

APP1 --> MON
APP2 --> MON
APP3 --> MON

APP1 --> DOS
APP2 --> DOS
APP3 --> DOS

APP1 --> UPC
APP2 --> UPC
APP3 --> UPC

Production Server Setup

1. Multi-Server Deployment

Server 1: Primary Application Server

# Server preparation
sudo useradd -m -s /bin/bash ff-cdn
sudo mkdir -p /opt/ff-cdn
sudo chown ff-cdn:ff-cdn /opt/ff-cdn

# Application deployment
sudo su - ff-cdn
cd /opt/ff-cdn
git clone https://github.com/FeelyFeely/ff-cdn.git .
python3.11 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Servers 2-3: Additional Application Servers

# Repeat the same process for additional servers
# Ensure identical configuration across all servers

2. Production Environment Configuration

# Production environment file
cat > .env << EOF
# Production environment
ENVIRONMENT=production
FLASK_ENV=production
DEBUG=False
SSL_VERIFY=True

# Production databases
MYSQL_HOST=prod-mysql-cluster.example.com
MYSQL_PORT=25060
MONGODB_MAIN_PROD=mongodb+srv://prod-user:[email protected]/gateway?authSource=admin&replicaSet=prod-replica&tls=true&tlsCAFile=/opt/ff-cdn/certs/mongodb-prod.crt

# Production storage
DO_USER_ASSET_SPACES_REGION=fra1
DO_USER_ASSET_SPACES_BUCKET_NAME=ff.user.assets.prod

# Performance settings
GUNICORN_WORKERS=8
GUNICORN_WORKER_CONNECTIONS=1000
GUNICORN_TIMEOUT=60

# Security settings
KC_URL=https://auth.feelyfeely.com/
SSL_CERT_PATH=/etc/ssl/certs/feelyfeely.com.crt
SSL_KEY_PATH=/etc/ssl/private/feelyfeely.com.key
EOF

3. Production Gunicorn Configuration

# gunicorn_config_prod.py
from multiprocessing import cpu_count
import os

# Server socket
bind = "127.0.0.1:8080"
backlog = 2048

# Worker processes
workers = max(2, cpu_count() * 2)
worker_class = "gevent"
worker_connections = 1000
max_requests = 1000
max_requests_jitter = 100
timeout = 60
keepalive = 5

# Process naming
proc_name = 'ff-cdn-prod'
default_proc_name = 'ff-cdn-worker'

# Server mechanics
daemon = False
pidfile = '/var/run/ff-cdn/ff-cdn.pid'
user = 'ff-cdn'
group = 'ff-cdn'
tmp_upload_dir = '/tmp'

# Logging
accesslog = '/var/log/ff-cdn/access.log'
errorlog = '/var/log/ff-cdn/error.log'
loglevel = 'info'
access_log_format = '%(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s" %(D)s'

# Process recycling
preload_app = True
max_requests_jitter = 50

# Security
limit_request_line = 4094
limit_request_fields = 100
limit_request_field_size = 8190

4. Production Systemd Service

sudo tee /etc/systemd/system/ff-cdn.service << EOF
[Unit]
Description=FeelyFeely CDN Production
After=network.target

[Service]
Type=notify
User=ff-cdn
Group=ff-cdn
WorkingDirectory=/opt/ff-cdn
Environment=PATH=/opt/ff-cdn/venv/bin
Environment=PYTHONPATH=/opt/ff-cdn
ExecStart=/opt/ff-cdn/venv/bin/gunicorn --config gunicorn_config_prod.py wsgi:app
ExecReload=/bin/kill -s HUP \$MAINPID
KillMode=mixed
TimeoutStopSec=5
PrivateTmp=true
Restart=always
RestartSec=3

# Security settings
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/ff-cdn /var/log/ff-cdn /var/run/ff-cdn /tmp

[Install]
WantedBy=multi-user.target
EOF

# Create required directories
sudo mkdir -p /var/log/ff-cdn /var/run/ff-cdn
sudo chown ff-cdn:ff-cdn /var/log/ff-cdn /var/run/ff-cdn

# Enable and start service
sudo systemctl daemon-reload
sudo systemctl enable ff-cdn
sudo systemctl start ff-cdn

Load Balancer Configuration

Nginx Configuration

# /etc/nginx/sites-available/ff-cdn
upstream ff_cdn_backend {
least_conn;
server 127.0.0.1:8080 max_fails=3 fail_timeout=30s;
server 127.0.0.1:8081 max_fails=3 fail_timeout=30s;
server 127.0.0.1:8082 max_fails=3 fail_timeout=30s;
}

# Rate limiting
limit_req_zone $binary_remote_addr zone=api:10m rate=100r/m;
limit_req_zone $binary_remote_addr zone=images:10m rate=1000r/m;

server {
listen 80;
server_name cdn.feelyfeely.com;
return 301 https://$server_name$request_uri;
}

server {
listen 443 ssl http2;
server_name cdn.feelyfeely.com;

# SSL configuration
ssl_certificate /etc/ssl/certs/feelyfeely.com.crt;
ssl_certificate_key /etc/ssl/private/feelyfeely.com.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384;
ssl_prefer_server_ciphers off;

# Security headers
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";

# Gzip compression
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript image/svg+xml;

# Client body size
client_max_body_size 100M;

# Timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;

# Cache static assets
location ~* \.(css|js|png|jpg|jpeg|gif|ico|svg)$ {
expires 1y;
add_header Cache-Control "public, immutable";
try_files $uri @backend;
}

# API endpoints with rate limiting
location /api/ {
limit_req zone=api burst=20 nodelay;
proxy_pass http://ff_cdn_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}

# Image endpoints with higher rate limits
location /image/ {
limit_req zone=images burst=100 nodelay;
proxy_pass http://ff_cdn_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

# Cache headers for images
add_header Cache-Control "public, max-age=31536000";
}

# Default backend
location @backend {
proxy_pass http://ff_cdn_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}

# Health check endpoint
location = /health {
proxy_pass http://ff_cdn_backend;
access_log off;
}

# Root location
location / {
proxy_pass http://ff_cdn_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}

Docker Deployment

Dockerfile

# Multi-stage Docker build
FROM python:3.11-slim as base

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
ENV DEBIAN_FRONTEND=noninteractive

# Install system dependencies
RUN apt-get update && apt-get install -y \
curl \
build-essential \
python3-opencv \
imagemagick \
libsm6 \
libxext6 \
libpq-dev \
&& rm -rf /var/lib/apt/lists/*

# Create application user
RUN useradd --create-home --shell /bin/bash app

# Set work directory
WORKDIR /app

# Copy requirements and install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY . .

# Copy certificates and assets
COPY ca-certificate.crt mongodb-1-cert.crt ./
COPY assets/ ./assets/

# Create necessary directories
RUN mkdir -p logs tmp && \
chown -R app:app /app

# Switch to non-root user
USER app

# Expose port
EXPOSE 8080

# Health check
HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8080/health || exit 1

# Default command
CMD ["gunicorn", "--config", "gunicorn_config.py", "wsgi:app"]

Docker Compose for Production

# docker-compose.prod.yml
version: '3.8'

services:
ff-cdn:
build: .
image: ff-cdn:latest
container_name: ff-cdn-app
restart: unless-stopped
ports:
- "8080:8080"
env_file:
- .env.prod
volumes:
- ./logs:/app/logs
- ./tmp:/app/tmp
- ./certs:/app/certs:ro
networks:
- ff-cdn-network
depends_on:
- redis
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s

redis:
image: redis:7-alpine
container_name: ff-cdn-redis
restart: unless-stopped
ports:
- "6379:6379"
volumes:
- redis_data:/data
networks:
- ff-cdn-network
command: redis-server --appendonly yes

nginx:
image: nginx:alpine
container_name: ff-cdn-nginx
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
- ./ssl:/etc/ssl:ro
- ./logs/nginx:/var/log/nginx
networks:
- ff-cdn-network
depends_on:
- ff-cdn

volumes:
redis_data:

networks:
ff-cdn-network:
driver: bridge

Docker Build and Deploy

# Build production image
docker build -t ff-cdn:latest .

# Tag for registry
docker tag ff-cdn:latest registry.digitalocean.com/feelyfeely/ff-cdn:latest

# Push to registry
docker push registry.digitalocean.com/feelyfeely/ff-cdn:latest

# Deploy with docker-compose
docker-compose -f docker-compose.prod.yml up -d

Platform Deployments

DigitalOcean App Platform

App Platform Spec (app.yaml)

name: ff-cdn
region: fra1

services:
- name: ff-cdn-api
source_dir: /
github:
repo: FeelyFeely/ff-cdn
branch: main
deploy_on_push: true

run_command: gunicorn --config gunicorn_config.py wsgi:app
environment_slug: python
instance_count: 3
instance_size_slug: basic-xxs

http_port: 8080

health_check:
http_path: /health
initial_delay_seconds: 60
period_seconds: 10
timeout_seconds: 5
success_threshold: 1
failure_threshold: 3

envs:
- key: ENVIRONMENT
value: production
- key: FLASK_ENV
value: production
- key: MYSQL_HOST
value: ${db-mysql.HOSTNAME}
- key: MYSQL_USER
value: ${db-mysql.USERNAME}
- key: MYSQL_PASSWORD
value: ${db-mysql.PASSWORD}
type: SECRET

databases:
- name: db-mysql
engine: MYSQL
version: "8"
size: db-s-1vcpu-1gb

static_sites:
- name: ff-cdn-docs
source_dir: /docs
github:
repo: FeelyFeely/ff-cdn
branch: main
routes:
- path: /docs

Heroku Deployment

Procfile

web: gunicorn --config gunicorn_config.py wsgi:app
worker: python worker.py

Heroku Configuration

# Create Heroku app
heroku create ff-cdn-prod

# Set environment variables
heroku config:set ENVIRONMENT=production
heroku config:set FLASK_ENV=production
heroku config:set DEBUG=False

# Set database URLs
heroku config:set DATABASE_URL="mysql://user:pass@host:port/db"
heroku config:set MONGODB_URL="mongodb://user:pass@host:port/db"

# Set storage configuration
heroku config:set DO_USER_ASSET_SPACES_KEY="your-key"
heroku config:set DO_USER_ASSET_SPACES_SECRET="your-secret"

# Deploy
git push heroku main

# Scale dynos
heroku ps:scale web=3 worker=1

Database Deployment

MySQL High Availability Setup

-- Create replication user
CREATE USER 'replication'@'%' IDENTIFIED BY 'replication_password';
GRANT REPLICATION SLAVE ON *.* TO 'replication'@'%';

-- Configure master server
SET GLOBAL server_id = 1;
SET GLOBAL log_bin = ON;
SET GLOBAL binlog_format = 'ROW';

-- Configure slave servers
SET GLOBAL server_id = 2; -- Different for each slave
CHANGE MASTER TO
MASTER_HOST='master-host',
MASTER_USER='replication',
MASTER_PASSWORD='replication_password',
MASTER_LOG_FILE='mysql-bin.000001',
MASTER_LOG_POS=0;

MongoDB Replica Set

// Initialize replica set
rs.initiate({
_id: "ff-cdn-replica",
members: [
{ _id: 0, host: "mongo1:27017" },
{ _id: 1, host: "mongo2:27017" },
{ _id: 2, host: "mongo3:27017" }
]
});

// Check replica set status
rs.status();

Monitoring and Logging

Production Logging Configuration

# logging_config.py
import logging
import logging.handlers
import os

def setup_logging():
"""Configure production logging"""

# Create logs directory
os.makedirs('/var/log/ff-cdn', exist_ok=True)

# Configure root logger
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

# Application logger
app_logger = logging.getLogger('ff-cdn')
app_handler = logging.handlers.RotatingFileHandler(
'/var/log/ff-cdn/app.log',
maxBytes=10485760, # 10MB
backupCount=5
)
app_handler.setFormatter(logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s'
))
app_logger.addHandler(app_handler)

# Error logger
error_logger = logging.getLogger('ff-cdn.errors')
error_handler = logging.handlers.RotatingFileHandler(
'/var/log/ff-cdn/error.log',
maxBytes=10485760,
backupCount=5
)
error_handler.setLevel(logging.ERROR)
error_handler.setFormatter(logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(pathname)s:%(lineno)d - %(message)s'
))
error_logger.addHandler(error_handler)

Health Monitoring

# monitoring.py
import psutil
import requests
from datetime import datetime

def check_system_health():
"""Check system health metrics"""
return {
'timestamp': datetime.utcnow().isoformat(),
'cpu_percent': psutil.cpu_percent(interval=1),
'memory_percent': psutil.virtual_memory().percent,
'disk_percent': psutil.disk_usage('/').percent,
'load_average': psutil.getloadavg(),
'connections': len(psutil.net_connections())
}

def check_service_health():
"""Check external service health"""
services = {
'uploadcare': 'https://api.uploadcare.com/api/info/',
'spaces': 'https://fra1.digitaloceanspaces.com'
}

results = {}
for service, url in services.items():
try:
response = requests.get(url, timeout=5)
results[service] = {
'status': 'healthy' if response.status_code == 200 else 'unhealthy',
'response_time': response.elapsed.total_seconds()
}
except Exception as e:
results[service] = {
'status': 'error',
'error': str(e)
}

return results

SSL/TLS Configuration

SSL Certificate Setup

# Using Let's Encrypt with Certbot
sudo apt-get install certbot python3-certbot-nginx

# Obtain SSL certificate
sudo certbot --nginx -d cdn.feelyfeely.com

# Auto-renewal setup
sudo crontab -e
# Add: 0 12 * * * /usr/bin/certbot renew --quiet

SSL Configuration Best Practices

# Strong SSL configuration
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384;
ssl_prefer_server_ciphers off;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
ssl_stapling on;
ssl_stapling_verify on;

# HSTS
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";

Backup and Recovery

Database Backup

#!/bin/bash
# backup.sh

BACKUP_DIR="/opt/backups/ff-cdn"
DATE=$(date +%Y%m%d_%H%M%S)

# Create backup directory
mkdir -p $BACKUP_DIR

# MySQL backup
mysqldump -h $MYSQL_HOST -u $MYSQL_USER -p$MYSQL_PASSWORD main > $BACKUP_DIR/mysql_main_$DATE.sql
mysqldump -h $MYSQL_HOST -u $MYSQL_USER -p$MYSQL_PASSWORD business > $BACKUP_DIR/mysql_business_$DATE.sql

# MongoDB backup
mongodump --uri="$MONGODB_MAIN_PROD" --out=$BACKUP_DIR/mongodb_$DATE

# Compress backups
tar -czf $BACKUP_DIR/ff-cdn_backup_$DATE.tar.gz $BACKUP_DIR/*_$DATE*

# Upload to storage
aws s3 cp $BACKUP_DIR/ff-cdn_backup_$DATE.tar.gz s3://ff-cdn-backups/

# Clean old backups (keep 30 days)
find $BACKUP_DIR -name "*.tar.gz" -mtime +30 -delete

Disaster Recovery

#!/bin/bash
# restore.sh

BACKUP_FILE=$1
RESTORE_DIR="/tmp/restore"

# Extract backup
mkdir -p $RESTORE_DIR
tar -xzf $BACKUP_FILE -C $RESTORE_DIR

# Restore MySQL
mysql -h $MYSQL_HOST -u $MYSQL_USER -p$MYSQL_PASSWORD main < $RESTORE_DIR/mysql_main_*.sql
mysql -h $MYSQL_HOST -u $MYSQL_USER -p$MYSQL_PASSWORD business < $RESTORE_DIR/mysql_business_*.sql

# Restore MongoDB
mongorestore --uri="$MONGODB_MAIN_PROD" --drop $RESTORE_DIR/mongodb_*

# Cleanup
rm -rf $RESTORE_DIR

Performance Optimization

Production Performance Tuning

# performance_config.py

# Gunicorn optimization
workers = min(32, (os.cpu_count() * 2) + 1)
worker_class = "gevent"
worker_connections = 1000
max_requests = 1000
max_requests_jitter = 100

# Database connection pooling
MYSQL_POOL_SIZE = 20
MYSQL_POOL_RECYCLE = 3600
MYSQL_POOL_TIMEOUT = 20

# Redis caching
REDIS_URL = "redis://localhost:6379/0"
CACHE_DEFAULT_TIMEOUT = 3600
CACHE_KEY_PREFIX = "ff-cdn:"

# Image processing optimization
UPLOAD_TIMEOUT = 60
PROCESS_TIMEOUT = 120
CONCURRENT_UPLOADS = 10

Troubleshooting Production Issues

Common Production Problems

High Memory Usage

# Check memory usage
free -h
ps aux --sort=-%mem | head

# Check application memory
sudo systemctl status ff-cdn
journalctl -u ff-cdn -f

# Restart if needed
sudo systemctl restart ff-cdn

Database Connection Issues

# Check database connectivity
python -c "from db_config import mysql; print(mysql.connection.open)"

# Check connection pool
netstat -an | grep :3306 | wc -l

# Monitor slow queries
tail -f /var/log/mysql/slow-query.log

Storage Issues

# Check disk space
df -h

# Check DigitalOcean Spaces connectivity
python -c "
from models.digitalocean.spaces import DigitalOceanSpaces
spaces = DigitalOceanSpaces('fra1', 'endpoint', 'key', 'secret')
print('Spaces accessible')
"

Emergency Procedures

Service Restart

# Graceful restart
sudo systemctl reload ff-cdn

# Full restart
sudo systemctl restart ff-cdn

# Check status
sudo systemctl status ff-cdn
journalctl -u ff-cdn -n 50

Rollback Deployment

# Git rollback
git checkout previous-stable-tag
git push origin main --force

# Docker rollback
docker pull registry.digitalocean.com/feelyfeely/ff-cdn:previous-tag
docker-compose -f docker-compose.prod.yml up -d

This completes the deployment guide covering all major deployment scenarios from development to production, including best practices for monitoring, security, and disaster recovery.