Weather Blog

Forecast models: Just really expensive dart boards?

Forecast models: Just really expensive dart boards?
Photo courtesy Flickr user "raveller"

Originally published Oct. 17, 2008:

We hear it all the time when someone comes into our weather office: "So, where's the dartboard?"

It's actually hidden in our coat closet, but more times than not the games of cricket are just for fun and we base our forecasts off computer model projections.

But computers are never wrong, right? So how do forecasts still go awry?

The way computer models work is that we take weather observations from around the globe from various sources -- such as weather instruments on the ground, ships at sea, weather balloons, satellites, pilot reports, etc. All that data then gets fed into the computer, and using what we know about how the planet and dynamics work, we apply incredibly complex mathematical equations to that data to try and figure out how the conditions right now will change over time.

 

The problem with forecast models is that the atmosphere is really, really big, and computers are still not yet fast enough to be able to account for its entirety.

Think about just how much air is on the planet -- the surface area of Earth is roughly 15.7 million miles and the troposphere, which is basically where our weather occurs, extends from the ground to roughly 40,000 feet (or roughly 7 1/2 miles) high. That's an incredible playground for weather to be created and move around.

To have a perfect computer model, we would need to know what is going on at every parcel of air at a given moment, have a perfect terrain map of every inch of the globe, and then have the perfect mathematical equations to calculate that data, and account for every nook and cranny on the planet that could affect the data.

You can see where this is going -- let's just say dartboard sales won't be tanking anytime soon.

First of all, we don't know what is going on at every parcel of air. You likely don't have a weather station transmitting the data right next to your head as you are reading this (unless you're truly a weather geek. But if so, I'll bet your neighbor doesn't. If he does, let me know of this neighborhood and if there is any real estate available.)

So what we do is take what observations we do have, and then extrapolate the data to fill in the gaps.

Second, we can't calculate every parcel of air, because even though nowadays it only takes a few seconds to download that Bee Gees song from iTunes to help you learn CPR, even the 'superest' of super computers still don't have enough processing power to calculate that data for every parcel of air and have a forecast for tomorrow ready before next summer.

What's done instead is to calculate the atmosphere at certain layers -- maybe 25-35 layers and extrapolate the rest.

The third challenge is that we can't use too high a terrain resolution because that would mean even more data for the computer to crunch. This was a particular challenge for the Northwest because older models had a difficult time accurately depicting the Olympic and Cascade mountains. But we are making great strides here, and while not perfect, resolutions are much better to where there are some local models available now that have a much better grasp of the terrain.

And finally, we still don't have the perfect mathematical equations to simulate the entire realm of weather, but we're pretty good.

So to recap -- we have to average initial data over areas where there are no observations, we have to average the entire atmosphere from specific layers, and we have to average terrain over fairly wide margins, and even then, our math might not be spot on perfect.

Northwest A Double Whammy

So you can see where problems might arise. What if there's some storm developing in an area where we don't have an observation? Let's say at Point A, it's sunny, and Point C, it's sunny, but there's no observation at Point B, where it's suddenly raining.

The model will look at A and C and assume B is sunny too. This creates a model error. But as the model computes the forecast out farther into the future, this error will become magnified and can ruin the accuracy of the whole thing.

For the Pacific Northwest, it's particularly daunting as we have two distinct challenges when it comes to forecast models.

One, almost all our weather comes off the Pacific Ocean, but there's hardly any observations out there, save for a few ships, pilots, and whales who have joined the information superhighway. So the prospect for observational errors is higher than other places inland where you have way more data available.

Second, our terrain is very complex, and the larger scale models have to "dumb down" the terrain in order to save computing power for processing a larger area. As I mentioned earlier, this is a particular problem if the Cascades and Olympics are not correctly factored in. Some of the national and continental models had resolutions of about 30 kilometers, which means the model would factor in one point every 30 km (18.6 miles). Now look at our Cascades which are, what, about 60-70 miles wide? That a lot of averaging. I remember they used to have it to where Seattle was at about 1,000 feet, and the topography was a slow slope up from the coast to 3,000 feet in the Cascades, and the Olympics didn't exist.

What's available now?

Technology has made weather forecaster's lives much easier. For one, the models are now in color and available on the Internet at the click of a mouse. When I first started here in the mid 90s, my first task each day was to spend 30 minutes grabbing the black and white print outs of all the charts off our really big printer, tear off each page, then attach each model for each forecast hour to individual clipboards on a big wall. (Then I'd have to walk home barefoot in the snow, uphill both ways.) It's much easier to see that big cold snap in blue than just having to read a number on a black line.

And as computers get faster, we can make better models. For example, the University of Washington now runs a few higher resolution models for just the Pacific Northwest that since it focuses on a smaller area, can use a better terrain model and can pick up nuances such as the Olympic Rain Shadow and Puget Sound Convergence Zone. That is one of our big lifelines today.

Other universities and agencies make similar regional models. In fact, there are several models out there. All it takes is a PhD in weather research, some fast computers, and a real grasp of atmospheric dynamics and you too can program a model.

But there are a few biggies -- the Americans run models called "GFS" and "NAM" and are in testing stages of replacing the GFS with a new "WRF". The Canadians have their own, as do the British (UKMET) and Europeans (ECMWF). Japan has one for Asia too. Some private companies have them too, as KOMO does through their weather computer software provider Weather Central. That's called ADONIS. That model also drives our MyKOMO4Cast product. There's a dozen or so models that focus solely on tropical weather and forecasting hurricanes. Here is a chart that shows hurricane forecast models tracks, courtesy of stormpulse.com

Overall, each model takes a slightly different approach -- perhaps using different altitudes for its layers, or perhaps puts different emphasis on certain observations, or uses slightly different equations.  You learn that some models do better in certain situations, and having more than one computer opinion can help -- especially when they're all on the same page. (When they wildly diverge, be it from model to model or from one time period to the next, it really lowers confidence in an extended forecast and raises the anxiety of weather forecasters. The exchange rate is two grey hairs for each model discrepency -- three if it's a snow forecast.)

But since models only have a few hours to crunch the data and get it published in a timely matter before the data is outdated, some models only run out a few hours or a few days and in exchange, use the leftover computer power to run a higher resolution for terrain.

Other, long-range models sacrifice resolution for being able to compute longer time frame. We have one model that goes out 16 days, but runs at an even lower resolution beyond day 7 as it hurries up to give just an overview of the last 9 days since odds are it'll change anyway. That's why we stop at 7 days on the extended forecast graphic.

Here's an example of today's model at Day 7:

 And Day 8: (Note how the rain blobs aren't as defined)

 

That's not to say it's not fun to look at these long range models. I wish I had saved yesterday's version as it had arctic air rolling in from interior B.C. and cold enough to snow in Seattle on Oct. 28 and 30, which it has since today given up on. But even so, it was a typical problem in that the model that far out runs at such a low resolution, it doesn't factor in the mountains and their blocking ability to where 9 times out of 10, that air stays east of the Rockies and Cascades.

It's one of our greater wintertime forecasting challenges in trying to account for this deficiency, but this is also why we still need humans to forecast :)

What happened Wednesday?

All this lead up was to show that they are not perfect, as was evidenced Wednesday night in Western Washington. Forecast models -- all of them -- had rain holding off in the Seattle area until well after midnight, and even into early Thursday morning. Instead, rain raced into the Seattle area during the evening commute.

Here is a look at what one forecast model, issued at 5 a.m. Wednesday, had for 5 p.m. Wednesday:

 

The colored blobs are expected rain. We're still shaking our heads at how far off it could be. Guess the model ate some bad sushi or something.

That just goes to show the challenges we face. Even with all our technology, we aren't perfect. As computers get faster and faster, weather forecasting accuracy should continue to improve as we can calculate even more data.

But we're keeping the dartboard in a safe, secure location... just in case :)