Pangu-Weather, an AI-based 3D high-resolution system developed by HUAWEI CLOUD AI team, offers precise weather forecasting. Leveraging 39 years of global weather data, the system provides faster and more accurate predictions, outperforming traditional numerical methods. This breakthrough, open to the meteorological community, can advance weather-dependent industries and aid global disaster preparation.
Dalith Steiger and Andy Fitze, Co-Founders of SwissCognitive, World-Leading AI Network – “Understanding the Weather Better – With Pangu-Weather, a 3D High-Resolution System for precise forecasting”
For centuries, the weather has fascinated mankind, as we have always been directly dependent on the elements. The awareness for the importance of weather and the preparation for various weather phenomena enters our minds once again when natural disasters occur. Recently, this has been the case with increasing frequency, and we must acknowledge that we are reliant on predictable weather conditions.
The importance of weather forecasting for business
For those earning their living in agriculture, this realization may sound almost laughable, as they constantly keep themselves informed on the forecast to take the best possible precautions for their livestock, crops and other produce. However, it is not only agriculture which is affected. Weather also has a direct impact on the energy industry, the construction industry, insurance companies and many more. But for all industries applies: Preparations are only as accurate and useful as the forecasts they are based on.
The accuracy of weather forecasts: Understanding complex systems
Unfortunately, weather forecasts are not as accurate as the global community wants them to be. This is exactly why the HUAWEI CLOUD AI team chose to focus on weather predictions. In the words of Dr. Tian Qi, Chief Scientist of HUAWEI CLOUD AI Field, an IEEE Fellow, and Academician of the International Eurasian Academy of Sciences: “Weather forecasting is one of the most important scenarios in the field of scientific computing because meteorological prediction is a very complex system, yet it is difficult to cover all aspects of mathematical and physical knowledge. […] (Huawei 2023). Let’s find out together, how existing forecasting systems can be strengthened.
Thank you for reading this post, don't forget to subscribe to our AI NAVIGATOR!
Methods to predict weather
At present, the weather is predicted by numerical methods (NWP). Numerical methods are highly sensitive to the initial time point from which a model assumes because there are many uncertainties no matter which starting point is chosen. Therefore, large errors can be embedded in forecasts after only a few days of predicting. This is why meteorologists use ensemble forecasts. They include several starting points, which are based on slightly different conditions. Thus, the error rate is reduced (LMU – Faculty of Physics o.J.). Even if the error rate is lowered, it cannot be completely eliminated. According to Bi et al. (2023, S. 541), one reason for this is the required computational overhead of NWP which “limits the amount of ensemble members that may be included in a model, hence weakening the diversity and accuracy of probabilistic weather forecasts.” The solution for this and other challenges in weather forecasting is Pangu-Weather, an AI-based system.
Pangu-Weather, an AI-based system
Pangu-Weather trains deep networks for fast and accurate numerical weather forecasting (Bi et al. 2023, S. 537). The use of Pangu-Weather allows meteorologists for example to apply their expertise to the model in order to control noise and as a result improve ensemble forecast while simultaneously reducing costs (Bi et al. 2023, S. 537).
The concept of using AI-based systems in weather forecasting is not entirely new to the field. Deep learning methods assume that complex relationships between input and output date can be conceptualized by abundantly training a model with data sets without fully understanding the underlying physical procedures. Methods of deep learning were first applied to problems of precipitation forecasting and based on radar data or satellite data (Bi et al. 2023, S. 541). But according to Bi et al. (2023, S. 533–535) 3D models like Pangu-Weather herald a breakthrough in terms of speed and accuracy in numerical weather forecasting. This is because they train deep networks in a revolutionizing way.
How does Pangu-Weather work?
Like other models based on deep learning methods, Pangu- Weather captures relationships between two points in time. The Pangu-Weather model is trained on 39 years of global weather data (Bi et al. 2023, S. 537). New about 3D models is, that they can capture these relationships in three dimensions instead of only two. This additionally allows relationships between atmospheric states at different pressure levels to be detected. (Bi et al. 2023, S. 533) This results in Pangu-Weather being at times more accurate and also faster compared to two-dimensional models in forecasting the weather. A prime use case is, that Pangu-Weather is more than 10,000-times faster than the world’s best NWP, the operational IFS of ECMWF when it comes to producing deterministic forecast results on reanalysis data. In this case, it achieves greater accuracy as well (Bi et al. 2023, S. 535–537). These findings apply to sustained forecasts, but also to the prediction of extreme weather phenomena (Bi et al. 2023, 533-334).
Let’s read from Dr. Tian Qi again and find out, what the future might hold in terms of possibilities: “At present, Pangu-Weather mainly completes the work of the forecast system, and its main ability is to predict the evolution of atmospheric states. Our ultimate goal is to build next-generation weather forecasting framework using AI technologies to strengthen the existing forecasting systems (Huawei 2023).
3D AI-based models: A breakthrough in weather forecasting
For us at SwissCognitive, this is fantastic news. The breakthroughs in AI are happening even faster than they were predicted! One can even say that the potential of AI models in weather forecasting has been vastly underestimated. Therefore it has not been taken seriously by many scientists around the world according to Sven Titz a journalist at the renowned newspaper NZZ (2023). This also had to do with the fact that developments in this area usually tend to be incremental and progress is made rather slow. So, people were just not expecting results of such magnitude! The fact that scientists speak of a “possible paradigm shift” or “imminent breakthrough” (Titz 2023), can be considered a great success for the HUAWEI CLOUD AI team. However, the success story continues: It has been the first time that employees of a Chinese technology company are the sole authors of a Nature paper, according to the publication’s own index (Huawei 2023).
Pangu- Weather: A competitive model
In the estimate of SwissCognitive, Pangu- Weather shows that 3D weather forecasting models are relevant real-world application cases. It is a competitive model with vast potential. An example from May of 2023 illustrates this in an outstanding way. Remember, when Typhoon Mawar caught the world’s attention as the strongest tropical cyclone of the year thus far? According to the China Meteorological Administration, Pangu-Weather accurately predicted the trajectory of Typhoon Mawar five days before it changed course in the eastern waters of the islands of Taiwan (Huawei 2023). The difference in terms of error rate compared to the IFS of ECMWF is impressively visualized across the data points, as shown in the figure below:
Source: (Bi et al. 2023)
Pangu- Weather: The future of weather forecasting
Not only the model’s accuracy is impressive, but also the way the findings are distributed. Commenting on the significance and quality of the research by HUAWEI CLOUD, the academic reviewers from Nature explained that not only is Pangu-Weather very easy to download and run, but that it is executed quickly on even a desktop computer. “This means that anyone in the meteorological community can now run and test these models to their hearts’ desire. What a great opportunity for the community to explore how well the model predicts specific phenomena. That’s going to help with progress in the field.” Another reviewer noted that “the results themselves are a significant step beyond previous results. This work will, in my opinion, make people reevaluate what forecasting models might look like in the future” (Huawei 2023).
It is precisely this open approach, making insights available to all and providing the necessary food for thought on how to rethink the future in a field, that determines the success of an approach in SwissCognitive’s view. This is impressively demonstrated by Pangu- Weather. This case is clear proof that when researched and used with ethical standards considered, AI can lead to the improvement of living conditions and, most notably, the economic upliftment of global communities.
References
Bi, Kaifeng; Xie, Lingxi; Zhang, Hengheng; Chen, Xin; Gu, Xiaotao; Tian, Qi (2023): Accurate medium-range global weather forecasting with 3D neural networks. In: Nature 619 (7970), S. 533–538. DOI: 10.1038/s41586-023-06185-3.
Huawei (2023): Prestigious science journal Nature publishes paper about Pangu Weather AI Model authored by HUAWEI CLOUD researchers. Meteorological model shows strong performance when compared with traditional prediction in speed and accuracy. Online verfügbar unter https://www.huawei.com/en/news/2023/7/pangu-ai-model-nature-publish, zuletzt aktualisiert am 06.07.2023, zuletzt geprüft am 28.07.2023.
LMU – Faculty of Physics (o.J.): Ensemble Vorhersagen und Vorhersagbarkeit. Online verfügbar unter https://www.meteo.physik.uni-muenchen.de/DokuWiki/doku.php?id=arbeitsgruppen:ensemble_vorhersagen, zuletzt geprüft am 22.07.2023.
Titz, Sven (2023): Umsturz bei der Wettervorhersage: Modelle mit künstlicher Intelligenz holen die herkömmlichen Vorhersagemodelle ein 2023, 05.07.2023. Online verfügbar unter https://www.nzz.ch/wissenschaft/wettervorhersage-mit-kuenstlicher-intelligenz-wird-konkurrenzfaehig-ld.1745565?reduced=true, zuletzt geprüft am 28.07.2023.
About the Authors:
Dalith Steiger is a serial entrepreneur and a global AI Strategist and Thought-Leader. She belongs to the top pioneering women in cognitive technologies and one of the top digital shapers and leading voices in the global AI ecosystem. Dalith was featured in Onalytica’s Who’s Who in AI report as a global key opinion leader. She was born in Israel, grew up in Switzerland, and studied mathematics and business informatics at the University of Zurich. With Andy Fitze she co-founded the award-winning AI start-up SwissCognitive, and the CognitiveValley Foundation. Dalith is a global AI-strategy advisor and speaker, sharing her extensive knowledge and experience in the field of AI around the world. She sits in several boards and juries, is leading the Swiss IT Leadership Forum, advises various companies in their AI journey, mentors young women and girls in tech, and teaches AI & Machine Learning in a CAS module at the Applied University of Luzern. Besides her drive for cognitive technologies, she is also a loving mother of two young women, a passionate mountain biker and a big fan of high-heel shoes.
Andy Fitze is a serial entrepreneur, digital cognitive strategist, AI influencer, and top global AI and digital transformation advisor for start-ups and enterprise boards. Andy was featured in Onalytica’s Who’s Who in AI report October 2021 and is one of the top digital shapers. With Dalith Steiger he co-founded the award-winning start-up SwissCognitive, and the CognitiveValley Foundation. He is president of the Swiss IT Leadership Forum, member of the Board of Directors of SwissICT. Andy sits in several Boards of Directors of various companies. Andy is a lecturer and Member of the Strategic Advisory Board at Bern University of Applied Sciences and is a lecturer at the ETH for CAS Architecture Digitization. Previously Andy worked as Group CIO of RUAG, and at PostFinance he was responsible for IT governance. He holds a degree in electrical engineering (HTL), an Executive MBA from the University of St. Gallen, and received the Swiss CIO Award for Best IT Manager in Switzerland in 2015. To share his 30 years of extensive knowledge and experience, Andy is often seen on global stages. He is also a passionate skipper on the oceans – providing him with a great balance for head and soul.
Photo credit: Andy Fitze
Pangu-Weather, an AI-based 3D high-resolution system developed by HUAWEI CLOUD AI team, offers precise weather forecasting. Leveraging 39 years of global weather data, the system provides faster and more accurate predictions, outperforming traditional numerical methods. This breakthrough, open to the meteorological community, can advance weather-dependent industries and aid global disaster preparation.
Dalith Steiger and Andy Fitze, Co-Founders of SwissCognitive, World-Leading AI Network – “Understanding the Weather Better – With Pangu-Weather, a 3D High-Resolution System for precise forecasting”
For centuries, the weather has fascinated mankind, as we have always been directly dependent on the elements. The awareness for the importance of weather and the preparation for various weather phenomena enters our minds once again when natural disasters occur. Recently, this has been the case with increasing frequency, and we must acknowledge that we are reliant on predictable weather conditions.
The importance of weather forecasting for business
For those earning their living in agriculture, this realization may sound almost laughable, as they constantly keep themselves informed on the forecast to take the best possible precautions for their livestock, crops and other produce. However, it is not only agriculture which is affected. Weather also has a direct impact on the energy industry, the construction industry, insurance companies and many more. But for all industries applies: Preparations are only as accurate and useful as the forecasts they are based on.
The accuracy of weather forecasts: Understanding complex systems
Unfortunately, weather forecasts are not as accurate as the global community wants them to be. This is exactly why the HUAWEI CLOUD AI team chose to focus on weather predictions. In the words of Dr. Tian Qi, Chief Scientist of HUAWEI CLOUD AI Field, an IEEE Fellow, and Academician of the International Eurasian Academy of Sciences: “Weather forecasting is one of the most important scenarios in the field of scientific computing because meteorological prediction is a very complex system, yet it is difficult to cover all aspects of mathematical and physical knowledge. […] (Huawei 2023). Let’s find out together, how existing forecasting systems can be strengthened.
Thank you for reading this post, don't forget to subscribe to our AI NAVIGATOR!
Methods to predict weather
At present, the weather is predicted by numerical methods (NWP). Numerical methods are highly sensitive to the initial time point from which a model assumes because there are many uncertainties no matter which starting point is chosen. Therefore, large errors can be embedded in forecasts after only a few days of predicting. This is why meteorologists use ensemble forecasts. They include several starting points, which are based on slightly different conditions. Thus, the error rate is reduced (LMU – Faculty of Physics o.J.). Even if the error rate is lowered, it cannot be completely eliminated. According to Bi et al. (2023, S. 541), one reason for this is the required computational overhead of NWP which “limits the amount of ensemble members that may be included in a model, hence weakening the diversity and accuracy of probabilistic weather forecasts.” The solution for this and other challenges in weather forecasting is Pangu-Weather, an AI-based system.
Pangu-Weather, an AI-based system
Pangu-Weather trains deep networks for fast and accurate numerical weather forecasting (Bi et al. 2023, S. 537). The use of Pangu-Weather allows meteorologists for example to apply their expertise to the model in order to control noise and as a result improve ensemble forecast while simultaneously reducing costs (Bi et al. 2023, S. 537).
The concept of using AI-based systems in weather forecasting is not entirely new to the field. Deep learning methods assume that complex relationships between input and output date can be conceptualized by abundantly training a model with data sets without fully understanding the underlying physical procedures. Methods of deep learning were first applied to problems of precipitation forecasting and based on radar data or satellite data (Bi et al. 2023, S. 541). But according to Bi et al. (2023, S. 533–535) 3D models like Pangu-Weather herald a breakthrough in terms of speed and accuracy in numerical weather forecasting. This is because they train deep networks in a revolutionizing way.
How does Pangu-Weather work?
Like other models based on deep learning methods, Pangu- Weather captures relationships between two points in time. The Pangu-Weather model is trained on 39 years of global weather data (Bi et al. 2023, S. 537). New about 3D models is, that they can capture these relationships in three dimensions instead of only two. This additionally allows relationships between atmospheric states at different pressure levels to be detected. (Bi et al. 2023, S. 533) This results in Pangu-Weather being at times more accurate and also faster compared to two-dimensional models in forecasting the weather. A prime use case is, that Pangu-Weather is more than 10,000-times faster than the world’s best NWP, the operational IFS of ECMWF when it comes to producing deterministic forecast results on reanalysis data. In this case, it achieves greater accuracy as well (Bi et al. 2023, S. 535–537). These findings apply to sustained forecasts, but also to the prediction of extreme weather phenomena (Bi et al. 2023, 533-334).
Let’s read from Dr. Tian Qi again and find out, what the future might hold in terms of possibilities: “At present, Pangu-Weather mainly completes the work of the forecast system, and its main ability is to predict the evolution of atmospheric states. Our ultimate goal is to build next-generation weather forecasting framework using AI technologies to strengthen the existing forecasting systems (Huawei 2023).
3D AI-based models: A breakthrough in weather forecasting
For us at SwissCognitive, this is fantastic news. The breakthroughs in AI are happening even faster than they were predicted! One can even say that the potential of AI models in weather forecasting has been vastly underestimated. Therefore it has not been taken seriously by many scientists around the world according to Sven Titz a journalist at the renowned newspaper NZZ (2023). This also had to do with the fact that developments in this area usually tend to be incremental and progress is made rather slow. So, people were just not expecting results of such magnitude! The fact that scientists speak of a “possible paradigm shift” or “imminent breakthrough” (Titz 2023), can be considered a great success for the HUAWEI CLOUD AI team. However, the success story continues: It has been the first time that employees of a Chinese technology company are the sole authors of a Nature paper, according to the publication’s own index (Huawei 2023).
Pangu- Weather: A competitive model
In the estimate of SwissCognitive, Pangu- Weather shows that 3D weather forecasting models are relevant real-world application cases. It is a competitive model with vast potential. An example from May of 2023 illustrates this in an outstanding way. Remember, when Typhoon Mawar caught the world’s attention as the strongest tropical cyclone of the year thus far? According to the China Meteorological Administration, Pangu-Weather accurately predicted the trajectory of Typhoon Mawar five days before it changed course in the eastern waters of the islands of Taiwan (Huawei 2023). The difference in terms of error rate compared to the IFS of ECMWF is impressively visualized across the data points, as shown in the figure below:
Source: (Bi et al. 2023)
Pangu- Weather: The future of weather forecasting
Not only the model’s accuracy is impressive, but also the way the findings are distributed. Commenting on the significance and quality of the research by HUAWEI CLOUD, the academic reviewers from Nature explained that not only is Pangu-Weather very easy to download and run, but that it is executed quickly on even a desktop computer. “This means that anyone in the meteorological community can now run and test these models to their hearts’ desire. What a great opportunity for the community to explore how well the model predicts specific phenomena. That’s going to help with progress in the field.” Another reviewer noted that “the results themselves are a significant step beyond previous results. This work will, in my opinion, make people reevaluate what forecasting models might look like in the future” (Huawei 2023).
It is precisely this open approach, making insights available to all and providing the necessary food for thought on how to rethink the future in a field, that determines the success of an approach in SwissCognitive’s view. This is impressively demonstrated by Pangu- Weather. This case is clear proof that when researched and used with ethical standards considered, AI can lead to the improvement of living conditions and, most notably, the economic upliftment of global communities.
References
Bi, Kaifeng; Xie, Lingxi; Zhang, Hengheng; Chen, Xin; Gu, Xiaotao; Tian, Qi (2023): Accurate medium-range global weather forecasting with 3D neural networks. In: Nature 619 (7970), S. 533–538. DOI: 10.1038/s41586-023-06185-3.
Huawei (2023): Prestigious science journal Nature publishes paper about Pangu Weather AI Model authored by HUAWEI CLOUD researchers. Meteorological model shows strong performance when compared with traditional prediction in speed and accuracy. Online verfügbar unter https://www.huawei.com/en/news/2023/7/pangu-ai-model-nature-publish, zuletzt aktualisiert am 06.07.2023, zuletzt geprüft am 28.07.2023.
LMU – Faculty of Physics (o.J.): Ensemble Vorhersagen und Vorhersagbarkeit. Online verfügbar unter https://www.meteo.physik.uni-muenchen.de/DokuWiki/doku.php?id=arbeitsgruppen:ensemble_vorhersagen, zuletzt geprüft am 22.07.2023.
Titz, Sven (2023): Umsturz bei der Wettervorhersage: Modelle mit künstlicher Intelligenz holen die herkömmlichen Vorhersagemodelle ein 2023, 05.07.2023. Online verfügbar unter https://www.nzz.ch/wissenschaft/wettervorhersage-mit-kuenstlicher-intelligenz-wird-konkurrenzfaehig-ld.1745565?reduced=true, zuletzt geprüft am 28.07.2023.
About the Authors:
Photo credit: Andy Fitze
Share this: