Skip to main content

Data Flow

Session Upload Flow (Fully Asynchronous)​

Overview: All file processing happens in background jobs to provide instant user feedback and eliminate database contention.

Frontend (Instant Response):
1. User uploads FIT/ZIP file(s)
   ↓
2. Flask receives multipart form data
   ↓
3. Files saved to temporary location
   ↓
4. Background job created and returned immediately (<1 second)
   ↓
5. User sees: "✓ Files uploaded successfully. Processing in background..."

Background Processing (No Polling):
6. JobManager spawns background thread
   ↓
7. For each file:
   ├── Extract ZIP files (if applicable)
   ├── Find all FIT files recursively
   ├── Check for duplicates
   ├── Validate FIT format
   ├── Convert FIT → Parquet
   ├── Store in GridFS
   └── Save metadata to sessionsFIT/sessionsParquet
   ↓
8. Statistics analysis job automatically queued
   ↓
9. For each session:
   ├── Calculate metrics (pace, HR zones, power, etc.)
   ├── Store in sessionStatistics
   ├── Check and update personal bests
   └── Recalculate critical speed if PBs improved
   ↓
10. Runner history processor updates aggregated metrics
   ↓
11. Background job banner disappears when complete
   ↓
12. UI automatically shows new sessions (next page load)

Key Features:

  • No polling: Frontend doesn't poll job status (eliminates "busy" database errors)
  • Instant feedback: Upload completes in less than 1 second from user perspective
  • Status visibility: Background job banner shows processing status
  • Automatic chaining: File processing → Statistics analysis → History update
  • Error resilience: Individual file failures don't block other files

Training Plan Generation Flow (Fully Asynchronous)​

Overview: Training plan generation runs in background jobs to prevent UI blocking during CPU-intensive optimization.

Frontend (Instant Response):
1. User requests plan generation
   ↓
2. Flask creates background job and returns immediately
   ↓
3. User sees: "Training plan generation started! Check background jobs banner for status."
   ↓
4. Background job banner appears (no polling)

Background Processing (No Polling):
5. JobManager spawns background thread
   ↓
6. TrainingPlanGenerator.generate_training_plan_async() spawns subprocess
   ↓
7. generate_training_plan.py performs differential evolution optimization
   ↓
8. Schema and workouts retrieved
   ↓
9. Plan blocks populated with progressive overload calculated
   ↓
10. Plan saved to trainingplan collection
   ↓
11. Job status updated to 'completed'
   ↓
12. Background job banner disappears when complete
   ↓
13. User refreshes or navigates to see new plan

Key Features:

  • No polling: Frontend doesn't poll job status (eliminates "busy" database errors)
  • Instant feedback: Request completes in less than 1 second from user perspective
  • Status visibility: Background job banner shows processing status
  • Subprocess isolation: CPU-intensive optimization runs in separate process
  • Matches session upload pattern: Consistent UX across all async operations

For detailed algorithm explanation, see Training Algorithm.