I found the University of Washington news on what they have done with their AI model quite interesting.
Briefly, weather forecasting is done today via very complex computing models that fully simulate as best as can be managed using the fastest biggest supercomputer available. I’m not kidding, the UK’s Met office announced in Feb this year that they are now spending £1.2 Billion to increase their existing hardware.
Once upon a time I worked for Cray writing software. The Met Office was in the building next door at the time (before they relocated) and more or less bankrolled Cray with the money they spent on hardware. Another big customer at the time was ECMWF.
What some researchers over in the University of Washington have done is to take a completely different approach. Instead of a complete simulation of everything, they load up a model with about 40 years worth of historical weather observations, then use an AI to pattern match and so …
“It sees a typical pattern, recognizes how it usually evolves and decides what to do based on the examples it has seen in the past 40 years of data.”
I don’t think this will replace what currently happens, and is not as accurate (yet), but it is still fascinating . It will be interesting to see where this goes next.
Below are the materials provided by the University and Washington, written by Hannah Hickey and published Dec 15, 2020.
AI model shows promise to generate faster, more accurate weather forecasts
Today’s weather forecasts come from some of the most powerful computers on Earth. The huge machines churn through millions of calculations to solve equations to predict temperature, wind, rainfall and other weather events. A forecast’s combined need for speed and accuracy taxes even the most modern computers.
The future could take a radically different approach. A collaboration between the University of Washington and Microsoft Research shows how artificial intelligence can analyze past weather patterns to predict future events, much more efficiently and potentially someday more accurately than today’s technology.
The newly developed global weather model bases its predictions on the past 40 years of weather data, rather than on detailed physics calculations. The simple, data-based A.I. model can simulate a year’s weather around the globe much more quickly and almost as well as traditional weather models, by taking similar repeated steps from one forecast to the next, according to a paper published this summer in the Journal of Advances in Modeling Earth Systems.
“Machine learning is essentially doing a glorified version of pattern recognition,” said lead author Jonathan Weyn, who did the research as part of his UW doctorate in atmospheric sciences. “It sees a typical pattern, recognizes how it usually evolves and decides what to do based on the examples it has seen in the past 40 years of data.”
Although the new model is, unsurprisingly, less accurate than today’s top traditional forecasting models, the current A.I. design uses about 7,000 times less computing power to create forecasts for the same number of points on the globe. Less computational work means faster results.
That speedup would allow the forecasting centers to quickly run many models with slightly different starting conditions, a technique called “ensemble forecasting” that lets weather predictions cover the range of possible expected outcomes for a weather event – for instance, where a hurricane might strike.
“There’s so much more efficiency in this approach; that’s what’s so important about it,” said author Dale Durran, a UW professor of atmospheric sciences. “The promise is that it could allow us to deal with predictability issues by having a model that’s fast enough to run very large ensembles.”
Co-author Rich Caruana at Microsoft Research had initially approached the UW group to propose a project using artificial intelligence to make weather predictions based on historical data without relying on physical laws. Weyn was taking a UW computer science course in machine learning and decided to tackle the project.
“After training on past weather data, the A.I. algorithm is capable of coming up with relationships between different variables that physics equations just can’t do,” Weyn said. “We can afford to use a lot fewer variables and therefore make a model that’s much faster.”
To merge successful A.I. techniques with weather forecasting, the team mapped six faces of a cube onto planet Earth, then flattened out the cube’s six faces, like in an architectural paper model. The authors treated the polar faces differently because of their unique role in the weather as one way to improve the forecast’s accuracy.
The authors then tested their model by predicting the global height of the 500 hectopascal pressure, a standard variable in weather forecasting, every 12 hours for a full year. A recent paper, which included Weyn as a co-author, introduced WeatherBench as a benchmark test for data-driven weather forecasts. On that forecasting test, developed for three-day forecasts, this new model is one of the top performers.
The data-driven model would need more detail before it could begin to compete with existing operational forecasts, the authors say, but the idea shows promise as an alternative approach to generating weather forecasts, especially with a growing amount of previous forecasts and weather observations.
Weyn is now a data scientist with Microsoft’s weather and finance division. This research was funded by the U.S. Office of Naval Research and a Department of Defense graduate fellowship.
The Paper
The actual paper itself is Open Access and can be found here – WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting