Shadow-Front Core Backend Architecture
1. Architecture Overview
Stack: Go 1.25, DDD hexagonal architecture, gRPC/Connect APIs
Storage: SQLite with SQLC type-safe queries, file-based video storage
Core Systems: Auth (RBAC + OTP), Clip management, Sprite generation, Download tickets
graph TB subgraph "API Layer" GRPC[gRPC/Connect Services] HTTP[HTTP Server] end subgraph "Application Layer" AS[AuthService] CS[ClipService] DS[DownloadService] end subgraph "Domain Layer" AUTH[Auth Domain] CLIP[Clip Domain] SHARED[Shared VOs] end subgraph "Infrastructure" REPO[Repositories] IDX[File Indexer] SPR[Sprite System] LOG[Logger] end subgraph "Storage" DB[(SQLite)] FS[File System] end GRPC --> AS GRPC --> CS GRPC --> DS HTTP --> SPR AS --> AUTH CS --> CLIP DS --> CLIP AUTH --> REPO CLIP --> REPO REPO --> DB IDX --> FS SPR --> FS CS --> IDX
2. Domain Layer (DDD)
Auth Domain
User Aggregate: ID, Username, Role (admin/user/viewer), linked accounts, active status
Session Entity: Token, browser fingerprint, IP tracking, TTL-based expiry
OTP System: Time-based codes, purpose-specific (login/settings/admin), attempt limiting
Clip Domain
Clip Aggregate: Game association, slug, path, duration, size, metadata, visibility rules
Game Entity: Name as natural key, cover art, description
Value Objects: GameName, ClipSlug, Visibility (public/private/hidden), VideoMetadata
Shared Domain
Core VOs: ID (UUID wrapper), FilePath (validation), Duration (ms), FileSize (bytes)
Events: UserCreatedEvent, ClipCreatedEvent, SpritesGeneratedEvent, VisibilityChangedEvent
Errors: Domain-specific typed errors for better error handling
3. Application Services
sequenceDiagram participant Client participant AuthService participant ClipService participant Repository participant EventBus Client->>AuthService: RequestOTP(username) AuthService->>Repository: CreateOTP() AuthService-->>Client: OTP Code Client->>AuthService: VerifyOTP(code) AuthService->>Repository: ValidateOTP() AuthService->>Repository: CreateSession() AuthService-->>Client: Session Token Client->>ClipService: CreateClip(metadata) ClipService->>Repository: Save(clip) ClipService->>EventBus: Publish(ClipCreatedEvent) ClipService-->>Client: Clip ID
AuthService: User CRUD, OTP generation/verification, session management
ClipService: Clip CRUD, game management, search, user assignment
DownloadService: Ticket generation, authorization, secure file delivery
4. Infrastructure Layer
Persistence (Repository Pattern)
SQLClipRepository: SQLC-generated queries, domain ↔ DB mapping
SQLUserRepository: User persistence, session management
SQLGameRepository: Game metadata, search capabilities
HTTP Server
Middleware Chain: Logging → CORS → Auth → Request handler
Static Serving: /browse, /view, /media endpoints for file access
API Routes: /api/v1/sprites/*, /api/v1/videos/* for dynamic content
Background Services
File Indexer: Periodic scan (5s default), new file detection, DB sync
Sprite Generator: Worker pool (2 workers), queue management, retry logic
Log Shipper: Circular buffer, batch shipping, retry with backoff
5. API Layer (gRPC/Connect)
graph LR subgraph "Proto Services" CORE[Core Service
Games, Clips] AUTH[Auth Service
Users, OTP] LOG[LogService
Log Management] DL[DownloadService
Tickets] end subgraph "Interceptors" AI[Auth Interceptor
ACL Check] end subgraph "Transport" HTTP[HTTP/2] WS[WebSocket] end CORE --> AI AUTH --> AI LOG --> AI DL --> AI AI --> HTTP AI --> WS
Core Service: ListGames, ListClips, GetClip, SearchGames, SearchClips, AssignClipToUser
Auth Service: CheckSetupStatus, InitialSetup, RequestOTP, VerifyOTP, CreateUser
DownloadService: CreateTicket (secure token), Authorize (validate + stream)
6. Sprite Generation System
stateDiagram-v2 [*] --> Detected: New Video Detected --> Queued: Add to Queue Queued --> Processing: Worker Available Processing --> Extracting: FFmpeg Extract Frames Extracting --> Assembling: Create Sprite Sheets Assembling --> Generating: Generate WebVTT Generating --> Caching: Update Cache Caching --> Complete: Save Metadata Complete --> [*] Processing --> Failed: Error Failed --> Retry: Retry Logic Retry --> Queued: Re-queue Failed --> [*]: Max Retries
Frame Extraction: FFmpeg @ 100ms intervals, optimal codec settings
Sprite Assembly: 10x10 grids, JPEG compression, aspect ratio preservation
WebVTT Generation: Timeline mapping, browser-native integration
7. Data Flow Patterns
Authentication Flow
Browser → OTP Request → Generate 6-digit → Store w/ fingerprint → Return code
Browser → Verify OTP → Validate fingerprint → Create session → JWT token
Browser → API Request → Validate token → Check permissions → Process request
Video Processing Flow
File appears → Indexer detects → Create clip record → Queue sprite job
Worker picks job → Extract frames → Build sprites → Generate VTT → Update DB
Client requests → Check cache → Serve sprites → Display preview
Download Flow
Request download → Create ticket → Store fingerprint → Return ticket ID
Authorize download → Validate ticket → Check permissions → Stream file
Track progress → Update audit log → Expire ticket → Complete
8. Database Schema
erDiagram
games ||--o{ clips : contains
users ||--o{ user_sessions : has
users ||--o{ otp_codes : generates
clips ||--o{ download_tickets : enables
user_sessions ||--o{ download_tickets : authorizes
games {
text name PK
text slug UK
text cover_art_url
text description
}
clips {
text id PK
text game FK
text path
int duration_ms
int size_bytes
bool has_sprites
text visibility
text assigned_user_id FK
}
users {
text id PK
text username UK
text role
bool is_active
timestamp created_at
}
user_sessions {
text id PK
text user_id FK
text browser_fingerprint
text jwt_token_hash
timestamp expires_at
}
otp_codes {
text id PK
text code
text user_id FK
text browser_fingerprint
int attempts
timestamp expires_at
}
download_tickets {
text id PK
text clip_id FK
text session_id FK
text browser_fingerprint
timestamp expires_at
}9. Security Architecture
Browser Fingerprinting: Canvas hash + WebGL + timezone for device identification
OTP System: 6-digit codes, 5-minute TTL, 5 attempt limit
Session Management: JWT tokens, secure HttpOnly cookies, automatic renewal
10. Performance Optimizations
Sprite Caching: LRU cache (100 entries), lazy loading, aggressive HTTP caching
Database Indexes: Composite indexes on (game, created_at), (visibility, game)
Worker Pool: Concurrent processing (2 workers), job prioritization, retry queue
11. Monitoring & Observability
Structured Logging: JSON format, log levels, contextual fields
Circular Buffer: 10k entry limit, memory-efficient, batch shipping
Audit Trail: User actions, resource access, security events
12. Configuration
Environment Variables: CORE_* prefix, fallback to defaults
Command Flags: Override env vars, -addr, -data-path, -log-level
Directory Structure: ./config/ (DB), ./generated/ (sprites), ./temp/ (processing)
13. Development Workflow
task setup # Install deps, generate code
task dev # Start dev server (:8080)
task generate # Regenerate proto + SQLC
task build # Production build
14. Error Handling
Domain Errors: Typed errors (ErrUserNotFound, ErrClipNotFound)
Repository Errors: SQL error mapping to domain errors
API Errors: Connect error codes with detailed messages
15. Future Considerations
Scalability: Move to PostgreSQL, distributed workers, CDN for sprites
Features: Real-time notifications, clip editing, social features
Security: 2FA support, rate limiting, IP allowlisting