Web Analytics

Spatiotemporal inhomogeneity of accuracy degradation in AI weather forecast foundation models: A GNSS perspective

Table I. Properties of (AI) weather forecast (foundation) models
Table II. Variables supported by different models
Table III. Effective pressure levels supported by different models

Why not include more AI models in this research?

         We are primarily interested in how these weather forecast foundation models can be innovatively applied to geodesy and satellite remote sensing. Tropospheric delay is a path-integrated variable, the more data records along the integration path, the better the result. Generally, at least 25 layers are needed to keep the error from insufficient vertical resolution within an acceptable range. Models with only 13 pressure levels are too coarse for this purpose and would introduce unacceptable errors. Including them in a comparison would therefore be unfair.
         We've consulted with the teams of Pangu, FengWu, and Fuxi, and the answers were consistent: increasing the pressure levels from 13 to 37 significantly raises computational resource demands but offers limited improvement in weather forecast accuracy. Therefore, we anticipate that for a long time to come, as long as there is no game-changing leap in GPU computing power, most follow-up research will not introduce 37-layer models. In other words, the current situation where only GraphCast and FengWu support the full 37 layers is likely to persist for some time.
         The reality has borne this out: subsequent models, such as FuXi, GenCast, NeuralGCM, ClimaX, Aurora, OneForecast, and Aardvark Weather, have not adopted support for 37 pressure levels. Even the AI version of IFS introduced by ECMWF (known as AIFS) only supports 13 levels.

Spatiotemporal inhomogeneity of accuracy degradation

Top performer model of different area and forecast steps

Abstract

The artificial intelligence (AI) weather forecast foundation models can infer and generate precise global atmospheric state forecasts on the user’s device and with speed over 10,000 times faster than the operational Integrated Forecasting System (IFS), and it is making increasingly significant contributions to geodetic applications represented by the Global Navigation Satellite System (GNSS). However, existing studies on the investigation of these AI models are typically carried out by concentrating on specific one or several meteorological events in certain regions or by comparison with physical models, and the evaluation results obtained in this manner are not comprehensive and universal. Additionally, we find that the results obtained by the foundation models through the “rollout” method for forecasting are not uniform in terms of time and space. This temporal and spatial inhomogeneity of accuracy and accuracy degradation are related to AI algorithms and attributes of training data, etc., but these characteristics have not been thoroughly explored and analyzed. In this study, we obtained the global forecast results of foundation models for 2022 and subsequently derived the GNSS tropospheric delay through numerical integration. We calculated the mean deviation, mean absolute error, and root mean square error of these data. Using these metrics, we analyzed the spatiotemporal inhomogeneity in the accuracy degradation of foundation models, represented by Huawei Cloud Pangu-Weather, Google DeepMind GraphCast, and Shanghai AI Lab FengWu. We evaluated how this inhomogeneity changes with forecast time and identified the best-performing models across different regions and forecast durations. From the results, we find that taking topography into account when training the model enhances its accuracy at high altitudes, and the facilitating influence between the high related atmospheric variables such as precipitation and water vapor. The contributions of this study are twofold: it serves as a valuable reference for geodetic and remote sensing users employing foundational models, and offers insights and case supports for AI practitioners aiming to develop more accurate models for weather forecasting.

BibTeX

@article{ding2025spatiotemporal, title={Spatiotemporal inhomogeneity of accuracy degradation in AI weather forecast foundation models: A GNSS perspective}, author={Ding, Junsheng and Chen, Wu and Chen, Junping and Wang, Jungang and Zhang, Yize and Bai, Lei and Wang, Yuyan and Mi, Xiaolong and Liu, Tong and Weng, Duojie}, journal={International Journal of Applied Earth Observation and Geoinformation}, volume={139}, pages={104473}, year={2025}, publisher={Elsevier} }

Total Visits: