System Architecture Overview

This document provides a comprehensive overview of the Event Discovery system architecture, including all components, data flows, and integration patterns.

🏗️ High-Level Architecture

graph TD
    subgraph "External Sources"
        EB[Eventbrite API]
        MU[Meetup API] 
        LI[LinkedIn API]
        CS[Custom Scrapers]
    end

    subgraph "MCP Layer"
        MCP1[Eventbrite MCP]
        MCP2[Meetup MCP]
        MCP3[LinkedIn MCP]
    end

    subgraph "Processing Layer"
        EP[Event Processor]
        GEO[Geocoding Service]
        VAL[Data Validator]
        NORM[Data Normalizer]
    end

    subgraph "Storage Layer"
        MONGO[(MongoDB Atlas)]
        CACHE[Redis Cache]
    end

    subgraph "API Layer"
        REST[REST API]
        AUTH[Authentication]
        RATE[Rate Limiting]
    end

    subgraph "Frontend Layer"
        NEXT[Next.js App]
        CHAT[Chat Interface]
        MAP[Interactive Map]
        UI[React Components]
    end

    subgraph "Infrastructure"
        DEPLOY[Deployment]
        MONITOR[Monitoring]
        LOGS[Logging]
    end

    EB --> MCP1
    MU --> MCP2
    LI --> MCP3
    CS --> EP

    MCP1 --> EP
    MCP2 --> EP
    MCP3 --> EP

    EP --> GEO
    EP --> VAL
    VAL --> NORM

    GEO --> MONGO
    NORM --> MONGO
    MONGO --> CACHE

    CACHE --> REST
    MONGO --> REST
    REST --> AUTH
    REST --> RATE

    REST --> NEXT
    NEXT --> CHAT
    NEXT --> MAP
    NEXT --> UI

    NEXT --> DEPLOY
    DEPLOY --> MONITOR
    MONITOR --> LOGS

    style EB fill:#e3f2fd
    style MU fill:#e3f2fd
    style LI fill:#e3f2fd
    style MONGO fill:#e8f5e8
    style NEXT fill:#fff3e0
    style MAP fill:#fce4ec

🔄 Data Flow Architecture

sequenceDiagram
    participant User
    participant NextJS as Next.js Frontend
    participant API as REST API
    participant Processor as Event Processor
    participant MCP as MCP Servers
    participant Geocoder
    participant MongoDB
    participant Cache as Redis Cache

    Note over User,Cache: Event Discovery Flow

    User->>NextJS: Search for events
    NextJS->>API: POST /api/events/search

    API->>Cache: Check cached results
    alt Cache Hit
        Cache-->>API: Return cached events
        API-->>NextJS: Event data
    else Cache Miss
        API->>Processor: Process search request

        par Fetch from multiple sources
            Processor->>MCP: Fetch Eventbrite events
            MCP-->>Processor: Eventbrite data
        and
            Processor->>MCP: Fetch Meetup events  
            MCP-->>Processor: Meetup data
        and
            Processor->>MCP: Fetch LinkedIn events
            MCP-->>Processor: LinkedIn data
        end

        Processor->>Geocoder: Convert addresses to coordinates
        Geocoder-->>Processor: Lat/lng coordinates

        Processor->>MongoDB: Store processed events
        MongoDB-->>Processor: Storage confirmation

        Processor->>Cache: Cache search results
        Processor-->>API: Processed event data
        API-->>NextJS: Event data
    end

    NextJS->>NextJS: Render interactive map
    NextJS-->>User: Display events with map pins

🧩 Component Architecture

Processing Pipeline

flowchart LR
    subgraph "Input"
        A[Raw Event Data]
    end

    subgraph "Validation"
        B[Schema Validation]
        C[Required Fields Check]
        D[Data Type Conversion]
    end

    subgraph "Enhancement"
        E[Geocoding]
        F[Duplicate Detection]
        G[Category Normalization]
    end

    subgraph "Storage"
        H[Database Insert]
        I[Index Update]
        J[Cache Update]
    end

    subgraph "Output"
        K[Standardized Event]
    end

    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G
    G --> H
    H --> I
    I --> J
    J --> K

    style A fill:#e1f5fe
    style K fill:#e8f5e8

MCP Integration Pattern

graph TB
    subgraph "MCP Server Instance"
        MC[MCP Core]
        AP[API Parser]
        RL[Rate Limiter]
        ER[Error Recovery]
    end

    subgraph "External Platform"
        API[Platform API]
        AUTH[Authentication]
        DATA[Event Data]
    end

    subgraph "Our Application"
        CLIENT[MCP Client]
        PROC[Event Processor]
        DB[(Database)]
    end

    CLIENT -->|JSON-RPC| MC
    MC --> AP
    AP --> RL
    RL --> AUTH
    AUTH --> API
    API --> DATA
    DATA --> ER
    ER --> MC
    MC -->|Standardized Data| CLIENT
    CLIENT --> PROC
    PROC --> DB

    style MC fill:#f3e5f5
    style CLIENT fill:#e8f5e8
    style DB fill:#fff3e0

🗄️ Database Architecture

MongoDB Collection Structure

erDiagram
    EVENTS ||--o{ EVENT_TAGS : has
    EVENTS ||--o{ EVENT_IMAGES : contains
    EVENTS }o--|| VENUES : located_at
    EVENTS }o--|| ORGANIZERS : organized_by

    EVENTS {
        ObjectId _id PK
        string event_id UK
        string title
        text description
        datetime start_datetime
        datetime end_datetime
        string timezone
        string platform
        boolean is_online
        decimal price
        string currency
        string category
        float latitude
        float longitude
        datetime scraped_at
        datetime updated_at
    }

    VENUES {
        ObjectId _id PK
        string name
        string address
        string city
        string postal_code
        float latitude
        float longitude
        string country
    }

    ORGANIZERS {
        ObjectId _id PK
        string name
        string profile_url
        text description
        string platform
        string external_id
    }

    EVENT_TAGS {
        ObjectId _id PK
        ObjectId event_id FK
        string tag
        string source
    }

    EVENT_IMAGES {
        ObjectId _id PK
        ObjectId event_id FK
        string image_url
        string alt_text
        string size
    }

Database Indexes

graph LR
    subgraph "Primary Indexes"
        PI1[event_id - Unique]
        PI2[_id - Primary]
    end

    subgraph "Search Indexes"
        SI1[title - Text]
        SI2[description - Text]  
        SI3[category - Single]
        SI4[platform - Single]
    end

    subgraph "Geospatial Indexes"
        GI1[location - 2dsphere]
        GI2[latitude_longitude - Compound]
    end

    subgraph "Time Indexes"
        TI1[start_datetime - Single]
        TI2[scraped_at - Single]
        TI3[city_date - Compound]
    end

    style PI1 fill:#e3f2fd
    style GI1 fill:#e8f5e8
    style TI1 fill:#fff3e0

🌐 Frontend Architecture

graph TB
    subgraph "Next.js App Router"
        APP[App Layout]
        HOME[Home Page]
        SEARCH[Search Page]
        EVENT[Event Detail]
    end

    subgraph "Components"
        MAP[Map Component]
        CHAT[Chat Interface]
        SEARCH_BAR[Search Bar]
        EVENT_CARD[Event Card]
        FILTERS[Filter Panel]
    end

    subgraph "State Management"
        CONTEXT[React Context]
        HOOKS[Custom Hooks]
        CACHE[SWR Cache]
    end

    subgraph "External Services"
        MAPS_API[Google Maps API]
        CHAT_API[Chat API]
        EVENTS_API[Events API]
    end

    APP --> HOME
    APP --> SEARCH
    APP --> EVENT

    HOME --> MAP
    HOME --> SEARCH_BAR
    SEARCH --> FILTERS
    SEARCH --> EVENT_CARD
    EVENT --> CHAT

    MAP --> CONTEXT
    CHAT --> HOOKS
    SEARCH_BAR --> CACHE

    MAP --> MAPS_API
    CHAT --> CHAT_API
    EVENT_CARD --> EVENTS_API

    style APP fill:#e3f2fd
    style MAP fill:#e8f5e8
    style CHAT fill:#fff3e0

🔒 Security Architecture

graph TB
    subgraph "Authentication Layer"
        JWT[JWT Tokens]
        OAUTH[OAuth Providers]
        API_KEY[API Keys]
    end

    subgraph "Authorization Layer"
        RBAC[Role-Based Access]
        PERMS[Permissions]
        SCOPE[API Scopes]
    end

    subgraph "Data Protection"
        ENCRYPT[Data Encryption]
        TLS[TLS/HTTPS]
        HASH[Password Hashing]
    end

    subgraph "Network Security"
        FIREWALL[Firewall Rules]
        RATE_LIMIT[Rate Limiting]
        IP_FILTER[IP Filtering]
    end

    subgraph "Database Security"
        DB_AUTH[DB Authentication]
        NET_ACCESS[Network Access Control]
        BACKUP[Encrypted Backups]
    end

    JWT --> RBAC
    OAUTH --> PERMS
    API_KEY --> SCOPE

    RBAC --> ENCRYPT
    PERMS --> TLS
    SCOPE --> HASH

    ENCRYPT --> FIREWALL
    TLS --> RATE_LIMIT
    HASH --> IP_FILTER

    FIREWALL --> DB_AUTH
    RATE_LIMIT --> NET_ACCESS
    IP_FILTER --> BACKUP

    style JWT fill:#ffebee
    style ENCRYPT fill:#e8f5e8
    style FIREWALL fill:#e3f2fd

📊 Performance Considerations

Caching Strategy

graph LR
    subgraph "Cache Layers"
        L1[Browser Cache]
        L2[CDN Cache]
        L3[API Cache]
        L4[Database Cache]
    end

    subgraph "Cache Types"
        STATIC[Static Assets]
        API_RESP[API Responses]
        QUERY[Query Results]
        SESSION[Session Data]
    end

    USER[User Request] --> L1
    L1 --> L2
    L2 --> L3
    L3 --> L4

    L1 --> STATIC
    L2 --> STATIC
    L3 --> API_RESP
    L4 --> QUERY

    style L1 fill:#e3f2fd
    style L3 fill:#e8f5e8
    style QUERY fill:#fff3e0

Scalability Patterns

graph TB
    subgraph "Horizontal Scaling"
        LB[Load Balancer]
        APP1[App Instance 1]
        APP2[App Instance 2]
        APP3[App Instance N]
    end

    subgraph "Database Scaling"
        PRIMARY[(Primary DB)]
        REPLICA1[(Replica 1)]
        REPLICA2[(Replica 2)]
    end

    subgraph "Microservices"
        EVENT_SVC[Event Service]
        GEO_SVC[Geocoding Service]
        SEARCH_SVC[Search Service]
        CACHE_SVC[Cache Service]
    end

    LB --> APP1
    LB --> APP2
    LB --> APP3

    APP1 --> EVENT_SVC
    APP2 --> GEO_SVC
    APP3 --> SEARCH_SVC

    EVENT_SVC --> PRIMARY
    GEO_SVC --> REPLICA1
    SEARCH_SVC --> REPLICA2

    EVENT_SVC --> CACHE_SVC
    GEO_SVC --> CACHE_SVC
    SEARCH_SVC --> CACHE_SVC

    style LB fill:#e3f2fd
    style PRIMARY fill:#e8f5e8
    style CACHE_SVC fill:#fff3e0

🚀 Deployment Architecture

graph TB
    subgraph "Development"
        DEV[Local Development]
        TEST[Testing Environment]
    end

    subgraph "CI/CD Pipeline"
        GIT[Git Repository]
        BUILD[Build Process]
        DEPLOY[Deployment]
    end

    subgraph "Production"
        FRONT[Frontend (Vercel)]
        API[API Server]
        DB[MongoDB Atlas]
        CACHE[Redis Cloud]
    end

    subgraph "Monitoring"
        METRICS[Metrics Collection]
        ALERTS[Alert System]
        LOGS[Log Aggregation]
    end

    DEV --> GIT
    TEST --> GIT
    GIT --> BUILD
    BUILD --> DEPLOY

    DEPLOY --> FRONT
    DEPLOY --> API
    API --> DB
    API --> CACHE

    FRONT --> METRICS
    API --> METRICS
    DB --> METRICS

    METRICS --> ALERTS
    METRICS --> LOGS

    style DEV fill:#e8f5e8
    style FRONT fill:#e3f2fd
    style METRICS fill:#fff3e0

This architecture provides a robust, scalable foundation for the Event Discovery application with clear separation of concerns and modern best practices.