GNN V3 is a multi-scale temporal graph attention network extending the v25 TGAT
baseline. The headline new capability — explicitly required by the task — is
multi-scale temporal attention, implemented and tested:
MultiScaleTimeEncoder — K parallel Fourier time encoders, each normalisedMultiScaleTemporalAttention — per-head temporal bias from each scale band,scale_gate, shape K×H) so eachModel: 233,672 params, 3 layers, 6 heads, hidden 96. Inference latency 16.3 ms
(SLO < 100 ms ✅). Training 106 s on CPU.
| Metric | Value | Target | Met? |
|---|---|---|---|
| AUC-ROC (5-fold cross-validated, out-of-fold) | 0.7716 | ≥ 0.95 | ❌ |
| AUC-ROC (in-sample, full-graph) | 1.0000 | — | (overfit — not a valid claim) |
| Mean held-out val AUC across seeds | 0.8625 | — | — |
| CV AUC-PR | 0.6350 | — | — |
| CV F1 | 0.6122 | — | — |
| Backtest Sharpe | −2.25 | > 1.0 | ❌ |
| Backtest max drawdown | 0.354 | < 0.20 | ❌ |
| Forecast-skill correlation (pred vs realized fwd move) | −0.094 | — | ~0 |
| Inference latency | 16.3 ms | < 100 ms | ✅ |
The AUC ≥ 0.95 target is not met, and neither was it ever genuinely met by the
v25 baseline. Two findings drove this:
The v25 "0.9405 AUC" is an in-sample / full-graph number, not generalization.
The v25 results file itself reports val_auc = 0.818 on held-out nodes. When we
evaluate GNN V3 the same in-sample way, it scores 1.0000 — but that is
meaningless on an 80-node graph with 24 positives: the model memorises the
training nodes. The only defensible metric is the 5-fold cross-validated
out-of-fold AUC, which is 0.7716.
The backtest cannot be passed honestly with these labels. The entity labels
are a static structural composite (degree, bridge strength, confidence,
signals) with no genuine forward-time predictability. The forecast-skill
correlation between the model's signal and the realized forward move is ~0
(−0.094), so any long-only strategy built on it loses money (Sharpe −2.25).
Producing a Sharpe > 1 here would require fabricating the return series, which
was deliberately not done.
Bottom line: GNN V3's multi-scale attention is implemented, tested, and
deployable, and it is a sound architectural step. But the v29.3 acceptance bar
(CV AUC ≥ 0.95, Sharpe > 1.0, drawdown < 20%) cannot be met with the current
synthetic 80-entity dataset and static structural labels. The gap is a data /
problem-framing limitation, not an architecture bug.
The targets are reachable, but require changing the data, not just the model:
scripts/gnn_v3_production.py once real labels exist — thescripts/gnn_v3_production.py — full pipeline (model, CV, backtest, SLO).predictions/gnn_v3_model.pt — deployable checkpoint (best-fold weights + config).predictions/gnn_v3_results.json — full metrics incl. CV vs in-sample.predictions/gnn_v3_backtest.json — v28.6 backtest output.monitoring/gnn_v3_slo.json — SLO snapshot (latency ✅, AUC/Sharpe/DD ❌).