Cross-Simulation Validation of an ML Surrogate for Multirotor Emergency-Landing Risk Assessment

초록

Machine-learning surrogates can accelerate expensive physics evaluators, but in safety-critical tasks, agreement with a training oracle is not sufficient to establish physical credibility. This limitation is especially important when surrogate predictions support operational decisions such as emergency-landing assessment. Standard held-out validation is limited be-cause surrogate predictions are compared against labels generated by the same analytical model used for training, so inherited model-form bias can remain hidden. To address this, we use a two-oracle evaluation framework that separates surrogate fidelity from independent credibility assessment. Oracle A is the analytical physics evaluator used to generate surrogate targets, and Oracle B is an independent PX4-Gazebo software-in-the-loop (SITL) simulator used only for external validation. In a multirotor emergency-landing case study, an XGBoost surrogate achieved strong held-out performance against Oracle A (R2 = 0.956). Independent SITL evaluation, however, revealed structured disagreement, including false hard-fails (11.80%) and missed failures (7.18%). Wind was more strongly associated with simulator safety margins than Oracle A risk, and a 2×2 wind- friction diagnostic further localized the mismatch. The results show that strong agreement with a parent oracle can remain scientifically fragile and that independent cross-simulation validation can expose hidden modeling assumptions. © 2026 IEEE.

키워드