Current Runtime Model

The active predictor is EnhancedBGPredictor in app/models.py. Runtime currently uses an LSTM + Attention sequence model. GRU is not active today; it is a planned challenger model.

Input:
  LOOKBACK rows x feature_count
  default LOOKBACK=864 = about 72 hours at 5-minute cadence

Model:
  LSTM(128, return_sequences=True)
  BatchNormalization
  Dropout(0.3)
  LSTM(64, return_sequences=True)
  LSTM(32) query branch
  LSTM(32) value branch
  Attention
  LSTM(64)
  Dense(PREDICTION_HORIZON_STEPS)

Output:
  one predicted BG value per future 5-minute step
  default PREDICTION_HORIZON_STEPS=18 = 90 minutes
  24 steps = 120 minutes

Training Pipeline

Training starts at /train, calls train_from_nightscout(), fetches Nightscout CGM/treatments/profile, normalizes BG to mmol/L, builds feature rows, creates sliding windows and trains with MAE loss.

• Fetch CGM entries for TRAINING_WINDOW_DAYS.
• Fetch treatments and the active Nightscout profile.
• Normalize timestamps to UTC, sort and de-duplicate CGM rows.
• Merge treatments into the CGM timeline and build IOB, COB, SMB, bolus, carbs and basal columns.
• Add profile, time and BG trend features.
• Build X as past LOOKBACK rows and y as future BG values.
• Validate on newest data using VALIDATION_DAYS or VALIDATION_SPLIT.
• Save timestamped Keras model, update models/latest.keras and write model metadata JSON.

Model Features

bg, iob, cob, smb, bolus, carbs,
isf, carb_ratio, basal, target,
hour_sin, hour_cos, wday_sin, wday_cos,
time_since_last_meal, time_since_last_bolus,
d_bg_5m, d_bg_15m, d_bg_30m,
current_bg

The most outcome-critical inputs are usually BG trend, IOB, COB, active target, recent bolus/carb timing, active ISF/CR/basal profile and whether AAPS/Loop devicestatus is fresh.

Prediction Pipeline

• /predict gets recent CGM, treatments over the DIA window and the active profile.
• Fallback IOB/COB is calculated from treatments.
• Fresh AAPS/Loop devicestatus can override IOB/COB and provide targets, autosens, dynamic ISF and predBG.
• The latest feature window is passed into the LSTM model.
• Bias correction is applied when enough completed prediction episodes exist.
• Degenerate/floor forecasts are detected and can fall back to cached last-good, AAPS or flat forecast.
• Prometheus metrics are updated for live BG, IOB/COB, forecasts, quality and fallbacks.
• A prediction episode is stored so future actual CGM can be compared against the forecast.

Outcome Metrics

Training quality and live outcome are measured separately. Training-time metrics tell if a new model looked good during validation; runtime metrics tell if predictions matched real future CGM.

Training:
  model_val_mae{horizon_min="30|60|90|120"}
  model_val_p95{horizon_min="30|60|90|120"}

Runtime:
  forecast_error{horizon_min="30|60|90|120"}
  prediction_mape{horizon_min="30|60|90|120"}
  recommendation_gate_mape_60
  recommendation_gate_episodes
  isf_bias_60m_by_hour_mmol{hour="00".."23"}
  prediction_bias_applied{horizon_min="30|60|90|120"}

MAE, MAPE, p95 and Bias

• MAE: average absolute error in mmol/L.
• MAPE: error relative to actual BG.
• p95: how bad the worst 5 percent of predictions are.
• Bias: whether actual BG is systematically higher or lower than predicted by hour.

Bias is calculated as actual - predicted at +60 minutes. Positive bias means DP under-predicted; negative bias means DP over-predicted. This drives bounded suggested ISF, carb ratio and basal schedule changes.

Safety and Recommendation Gate

Therapy hints are blocked when quality is not good enough. The gate can require enough completed episodes, acceptable +60m MAPE, fresh CGM and a non-degenerate forecast.

RECOMMENDATION_GATE_ENABLE=true
RECOMMENDATION_GATE_WINDOW_DAYS=7
RECOMMENDATION_GATE_MIN_EPISODES=30
RECOMMENDATION_GATE_MAX_MAPE60=0.30

Prediction Health Metrics

prediction_floor_hits_60min
prediction_degenerate
prediction_using_cached
prediction_using_aaps_fallback
prediction_using_flat_fallback
prediction_input_nan_cells
prediction_input_constant_cols
data_cgm_age_seconds
data_aaps_age_seconds
data_cgm_missing_total
data_treatments_skipped_total

Best Next Improvements

• Champion/challenger model selection before replacing a working model.
• Add GRU as a real challenger with MODEL_ARCH=lstm|gru.
• Add TCN/Conv1D/WaveNet-style challenger for fast time-series inference.
• Compare LSTM, AAPS/Loop forecast, flat current-BG and trend baseline in an ensemble selector.
• Add prediction uncertainty/confidence and use it to harden the recommendation gate.
• Weight meals, boluses, SMB, lows, highs and rapid BG movement more heavily during training.

LLMs are useful for explanation and reporting, but not as the numeric BG forecast model. The prediction path should stay in measurable time-series logic with MAE, MAPE, p95, bias and explicit safety gates.