Skip to main content

Architecture Overview

This document provides a comprehensive overview of the FeelyFeely CDN architecture, including system design, components, data flow, and scalability considerations.

High-Level Architecture​

graph TB
    subgraph "Client Layer"
        CA[Client Applications]
        WA[Web Applications]
        MA[Mobile Apps]
    end
    
    subgraph "CDN Layer"
        LB[Load Balancer]
        API[Flask API Server]
        GU[Gunicorn Workers]
    end
    
    subgraph "Processing Layer"
        UC[Uploadcare Service]
        WM[Watermark Processor]
        CV[OpenCV Engine]
    end
    
    subgraph "Storage Layer"
        DS[DigitalOcean Spaces]
        UAS[User Asset Storage]
        MDS[Mobile Device Storage]
        VPS[Vehicle Photo Storage]
    end
    
    subgraph "Database Layer"
        MY[MySQL Database]
        MG[MongoDB Instance]
        KC[Keycloak Auth]
    end
    
    CA --> LB
    WA --> LB
    MA --> LB
    LB --> API
    API --> GU
    API --> UC
    API --> WM
    WM --> CV
    API --> DS
    DS --> UAS
    DS --> MDS
    DS --> VPS
    API --> MY
    API --> MG
    API --> KC

System Components​

1. Application Layer​

Flask Application (app.py)​

  • Role: Core application factory
  • Features: CORS enabled, modular blueprint registration
  • Configuration: Development and production modes

Server Module (server.py)​

  • Role: Main entry point and route definitions
  • Features: Default endpoints, blueprint registration
  • Endpoints: Health check, service information

WSGI Gateway (wsgi.py)​

  • Role: Production server interface
  • Features: Gunicorn compatibility, worker process management

2. Database Layer​

MySQL Database (db_config.py, dbutils/mysql.py)​

  • Primary Database: Main application data
  • Business Database: Business-specific data
  • Features:
    • Connection pooling
    • Dictionary cursor support
    • Multi-database architecture
    • SQLAlchemy integration

MongoDB (dbutils/mongodb.py)​

  • Role: Document storage and flexible data models
  • Features:
    • SSL/TLS encrypted connections
    • Aggregation pipeline support
    • Upsert operations
    • Join operations across collections
    • Error handling and logging

3. Storage Layer​

DigitalOcean Spaces (models/digitalocean/spaces.py)​

  • Architecture: S3-compatible object storage
  • Buckets:
    • ff.user.assets: User-uploaded content
    • ff.organization.assets: Organization assets
    • ff.mobiledevice.photos: Mobile device photos
    • ff.vehicle.photos: Vehicle images

Features:

  • Presigned URL generation
  • File existence checking
  • Multi-region support
  • Automatic retry mechanisms

4. Image Processing Pipeline​

Uploadcare Integration (models/image/uploadcare/)​

  • Primary Processor: Cloud-based image processing
  • Features:
    • File upload and storage
    • Real-time transformations
    • Format optimization
    • Quality adjustment
    • CDN distribution

Watermark Engine (models/image/watermark/watermark_embed.py)​

  • Technology: OpenCV + Pillow
  • Intelligence: Brightness-based logo selection
  • Features:
    • Critical area detection (faces, text, edges)
    • Optimal positioning algorithm
    • Real-time processing
    • Multiple logo support

Data Flow Architecture​

1. Image Request Flow​

sequenceDiagram
    participant C as Client
    participant API as Flask API
    participant DO as DigitalOcean Spaces
    participant UC as Uploadcare
    participant WM as Watermark Engine
    
    C->>API: GET /image/{bucket}/{uuid}/{operation}/{file}
    API->>DO: Check if processed image exists
    
    alt Image exists
        DO->>API: Return processed image
        API->>WM: Apply watermark
        WM->>API: Return watermarked image
        API->>C: Return final image
    else Image doesn't exist
        API->>DO: Fetch original image
        DO->>API: Return original image
        API->>UC: Upload for processing
        UC->>UC: Apply transformations
        UC->>API: Return processed image
        API->>DO: Store processed image
        API->>WM: Apply watermark
        WM->>API: Return watermarked image
        API->>C: Return final image
    end

2. Database Operations Flow​

graph LR
    subgraph "Application Logic"
        AL[API Endpoints]
    end
    
    subgraph "Database Abstraction"
        MYSQL[MySQL Utils]
        MONGO[MongoDB Utils]
    end
    
    subgraph "Database Connections"
        MYC[MySQL Connection]
        MGC[MongoDB Connection]
    end
    
    subgraph "Data Storage"
        MYD[(MySQL Databases)]
        MGD[(MongoDB Collections)]
    end
    
    AL --> MYSQL
    AL --> MONGO
    MYSQL --> MYC
    MONGO --> MGC
    MYC --> MYD
    MGC --> MGD

Scalability Architecture​

1. Horizontal Scaling​

Worker Process Scaling​

# gunicorn_config.py
workers = (2 * cpu_count()) + 1
worker_class = 'gevent'
worker_connections = 1000

Database Scaling​

  • MySQL: Master-slave replication support
  • MongoDB: Replica set configuration
  • Connection Pooling: Efficient resource utilization

2. Vertical Scaling​

Resource Optimization​

  • Memory: Efficient image processing buffering
  • CPU: Multi-threaded image operations
  • I/O: Asynchronous file operations

3. Caching Strategy​

Multi-Level Caching​

graph TD
    subgraph "Caching Layers"
        CDN[CDN Edge Cache]
        APP[Application Cache]
        DB[Database Cache]
        FS[File System Cache]
    end
    
    CLIENT[Client Request] --> CDN
    CDN --> APP
    APP --> DB
    DB --> FS
    FS --> STORAGE[Storage Backend]

Security Architecture​

1. Authentication & Authorization​

Keycloak Integration​

  • Single Sign-On: Centralized authentication
  • JWT Tokens: Stateless authentication
  • Role-Based Access: Fine-grained permissions
graph LR
    CLIENT[Client App] --> KC[Keycloak Server]
    KC --> JWT[JWT Token]
    JWT --> API[CDN API]
    API --> AUTHZ[Authorization Check]
    AUTHZ --> RESOURCE[Protected Resource]

2. Data Protection​

Transport Security​

  • HTTPS/TLS: All communications encrypted
  • Certificate Management: Automated cert renewal
  • MongoDB SSL: Database connections secured

Storage Security​

  • Bucket Policies: Restricted access controls
  • Presigned URLs: Time-limited access
  • Encrypted Storage: Data at rest encryption

3. Input Validation​

Request Sanitization​

# Example from validation utilities
class Validation:
    def sanitize_filename(self, filename):
        # Remove dangerous characters
        # Validate file extensions
        # Check file size limits

Performance Architecture​

1. Image Processing Optimization​

Processing Pipeline​

  • Lazy Loading: Process images on-demand
  • Caching: Store processed versions
  • Format Optimization: Automatic format selection
  • Quality Adjustment: Dynamic quality levels

2. Database Optimization​

Query Optimization​

  • Indexed Queries: Optimized database indexes
  • Connection Pooling: Efficient resource usage
  • Query Caching: Reduce database load

3. Network Optimization​

Content Delivery​

  • Edge Caching: Global content distribution
  • Compression: Automatic image compression
  • HTTP/2: Multiplexed connections

Monitoring & Observability​

1. Application Monitoring​

Health Checks​

@app.route('/health')
def health_check():
    return {
        "status": "healthy",
        "services": {
            "mysql": check_mysql_connection(),
            "mongodb": check_mongodb_connection(),
            "spaces": check_spaces_connection()
        }
    }

2. Performance Metrics​

Key Performance Indicators​

  • Response Time: API endpoint latency
  • Throughput: Requests per second
  • Error Rate: Failed request percentage
  • Cache Hit Ratio: Caching efficiency

3. Logging Strategy​

Structured Logging​

import logging

logging.basicConfig(level=logging.ERROR)
logger = logging.getLogger(__name__)

# Error tracking with context
logger.error("Image processing failed", extra={
    "uuid": image_uuid,
    "operation": operation,
    "error_type": type(e).__name__
})

Deployment Architecture​

1. Environment Configuration​

Multi-Environment Support​

  • Development: Local development setup
  • Staging: Pre-production testing
  • Production: High-availability deployment

2. Infrastructure as Code​

Container Strategy​

# Multi-stage Docker build
FROM python:3.11-slim as base
# System dependencies installation
# Application setup and optimization

3. CI/CD Pipeline​

Deployment Flow​

graph LR
    DEV[Development] --> TEST[Testing]
    TEST --> BUILD[Build Image]
    BUILD --> DEPLOY[Deploy to Staging]
    DEPLOY --> VALIDATE[Validation Tests]
    VALIDATE --> PROD[Production Deploy]

Configuration Management​

1. Environment Variables​

Configuration Categories​

  • Database: Connection strings and credentials
  • Storage: DigitalOcean Spaces configuration
  • Processing: Uploadcare API keys
  • Authentication: Keycloak settings

2. Secrets Management​

Security Best Practices​

  • Environment Separation: Different keys per environment
  • Key Rotation: Regular credential updates
  • Access Control: Limited secret access

Error Handling & Recovery​

1. Fault Tolerance​

Resilience Patterns​

  • Circuit Breakers: Prevent cascade failures
  • Retry Logic: Automatic retry with backoff
  • Graceful Degradation: Fallback mechanisms

2. Disaster Recovery​

Backup Strategy​

  • Database Backups: Automated daily backups
  • Storage Replication: Cross-region redundancy
  • Configuration Backups: Infrastructure state preservation

Future Architecture Considerations​

1. Scalability Enhancements​

Microservices Migration​

  • Service Decomposition: Separate processing services
  • API Gateway: Centralized routing and authentication
  • Event-Driven Architecture: Asynchronous processing

2. Performance Improvements​

Advanced Caching​

  • Redis Integration: Distributed caching layer
  • Edge Computing: Closer processing to users
  • ML-Based Optimization: Intelligent image processing

3. Technology Upgrades​

Modern Stack Integration​

  • Kubernetes: Container orchestration
  • Service Mesh: Advanced networking
  • Observability Tools: Enhanced monitoring