Numerical Weather Prediction Questions Answered

[dtk]

Instead of derailing multiple threads, I thought it might be good to have a separate thread for questions and related discussion on numerical weather prediction.  I'll start with a few from a recent thread.

 

All good points Bob, but wild swings while getting closer say that there's more to it than just chaos. Take an exponential function as an example. If you equate its growth with error, then the growth becomes larger in time, thus the error becomes larger in time. But the one thing it does not do is grow erratic in time. The swings in models back and forth can't be explained just by chaos. If you proceed on that assumption then the gfs swing from 12z yesterday to 18z yesterday has to be blamed on the error in the 12z modeling of the initial conditions used for the 18z run. Now I don't think the 12z forecast at 6 hours was so bad that it caused such completely different scenario that in turn caused the 18z run to be so different. Accumulated data has to play a role. The non pros here are told all the time that the data significance is over played, but if that is true I need two things cleared up for me. One, do we use different data collection techniques over the US than we do over the Pacific (I think I already know that answer) and if yes, why if it's not that critical to a model outcome? Two, why do we collect data so frequently, again if it's not that critical.

 

A few things.  For one, data assimilation is an incremental, cumulative process.  You are exactly right in that observations are combined with a short term model forecast in as optimal way as possible.  In your example, if no observations were assimilated into the 18Z cycle, the 18Z forecast would be identical to the 12Z forecast.  There are some technicalities such as the use of later data cut-offs and a catch-up cycle that render this to be not exactly true, but from a conceptual point of view, it is.

 

To your specific questions:  What do you mean by "data collection techniques" and "why do we collect so frequently if not critical"?

 

There is such a huge variety of atmospheric observations that are continuously collected.  While true that things like radiosondes and surface metars are recorded with a certain cadence, most observations are actually quasi-continuous and/or with a much higher temporal frequency.  This is true for radars, satellite sounders, gps radio occultation, etc. Here is an example of observations assimilated within a +/- 3 hour window around 12Z for metars, ships, buoys, radiosondes, satellite AMVs, aircraft, radar winds, wind profilers, pibals, and scatterometer winds:

 

 

Here is a view of the satellite coverage for the polar orbiters for that same period color coded by satellite:

 

Ignore the bottom right panel for the geo satellites as there is a bunch of stuff missing.  Keep in mind that each of these satellites has a variety of sensors on them, some of which actually have thousands of IR channels to get information from different parts of the vertical (AIRS, IASI, and CRIS).  All of the polar orbiters have a MW sounder with 15-22 channels which are critical for NWP.

 

There are millions of observations that are assimilated into a single cycle.  ECMWF has a pretty good page for looking at data distributions that go into their cycles:

http://www.ecmwf.int/en/forecasts/charts/monitoring/dcover

 

For example, here is a plot from their page showing the coverage of another type of satellite observations:  gps radio occultation:

 

[dtk]

The atmosphere is more than math

With the storage and capability of computers it would be possible to scan into it for a 5+ day forecast, and note I say forecast as I think models are more aligned toward projecting possibilities rather than forecasting, the following:

Moderate+ Nino, low pressure developing tx/la gulf coast, -nao, +pna, month of February, and have all of those parameters combined into a visual graphic as to what has been the dominant result in the past The exacta set of data input I may not be addressing correctly but the method concept would work better than what we have now

This is nothing more than a variant of analog methods.  Statistical methods and analogs have a long, successful history in weather and seasonal forecasting.  However, this is a supplement to, not replacement of, the use of dynamical models and their ensembles. 

 

Small errors early in the run can balloon into big differences.  In addition to small errors in the initial conditions, every model is an approximation that introduces new errors at every step along the way.  The good news, as dtk has previously pointed out, is that we've become much better at determining the initial condtions (data assimilation).  In addition to ground observations, we have weather balloon data and extensive satellite data.  One of the biggest problems these days is that sometimes dtk makes a typo when entering the latest satellite data.  That can throw a whole run off.  It's why the 00z run of the GFS, for which data is generally entered after he's had dinner, is better than the 18z run, for which data entry usually occurs before he's had lunch.

It's true, small initial condition errors grow rapidly...probably exponentially early on and then leveling off to some saturation whereby requiring a logistic error growth model.  However, there are complex, nonlinear, chaotic components to the atmosphere meaning that error growth isn't necessarily clean.  The Lorenz 3-variable model is a good example demonstration how small errors can force trajectories on completely different paths beyond bifurcation points (see content within here: https://en.wikipedia.org/wiki/Lorenz_system).

 

So for short range forecasts of the large scale, we can expect to reduce forecast error as we decrease the lead time.  However, for longer range forecasts, this paradigm may not hold true depending on these complex, nonlinear, chaotic interactions.  This may also not hold true for individual features of interest that have less inherent predictability.  

 

Your bits in bold are too funny and you give me way too much credit.  I do not do operational NWP anymore....

[dtk]

I think what he is saying is to use analogs basically. I think there is merit to that idea and have ask the same question myself in the past. In addition to, not in place of, the math/physics being used, use prior experiences. I have no idea how you would do that, but if your data set is large enough you'd be able to find prior dates with similar conditions. So you incorporate what happened prior into the output of the model. It would take a massive set of data to be effective. I doubt we have a set of data large enough.

In addition to the techniques of analogs, there is something call "reforecasting"

http://journals.ametsoc.org/doi/abs/10.1175/BAMS-87-1-33

 

This at least does a level of calibration that tries to account for some of the system errors.

[Bob Chill]

Really excellent stuff, dtk. It takes an enthusiast a long time to grasp how immensely complex NWP really is and how they have become basically a modern miracle irt longer range forecasting. It took me at least 5 years of pretty avid model watching/storm tracking to break out of weather movie type of analysis and start using my brain to think beyond verbatim interpretation.

Models have improved dramatically in just the 10 years I've been regularly looking at them.

[SomeguyfromTakomaPark]

I've always wondered what people were talking about with parametrization.  

 

Wikipedia tells me that it refers to processes that aren't explicitly resolved by a model and it gives some examples "....the descent rate of raindrops, convective clouds, simplifications of the atmospheric radiative transfer on the basis of atmospheric radiative transfer codes, and cloud microphysics." 

 

I'm wondering how different these parameters can be from model to model and how large a role they play in the differences we see between solutions.  

 

Thanks for starting this thread!  Very cool idea.

[clskinsfan]

What an incredible thread. I had some questions a couple of weeks back about the ensembles. I was wondering what was different upon initialization to cause them to end at different conclusions. Or are they all run with exactly the same data and the differences are from chaos and slight tweaks throughout the run.

[WinterWxLuvr]

Great stuff dtk.

[WinterWxLuvr]

What an incredible thread. I had some questions a couple of weeks back about the ensembles. I was wondering what was different upon initialization to cause them to end at different conclusions. Or are they all run with exactly the same data and the differences are from chaos and slight tweaks throughout the run.

Yeah, great question about the ensembles. I see the word perturbed and would love to dtk explain that via an example.

[CAPE]

Yeah, great question about the ensembles. I see the word perturbed and would love to dtk explain that via an example.

Same. I think I get that perturbed implies different initial conditions(in order  to minimize uncertainty) but do the individual members use different mathematical equations(physics)?

[Bob Chill]

Same. I think I get that perturbed implies different initial conditions(in order to minimize uncertainty) but do the individual members use different mathematical equations(physics)?

IIRC the ensembles run the exact same physics. They would have to I think. If they ran different physics AND different initial conditions it would result in additional uncertainty.

I do know the control run is the same initial conditions as the op but lower resolution.

[CAPE]

IIRC the ensembles run the exact same physics. They would have to I think. If they ran different physics AND different initial conditions it would result in additional uncertainty.

I do know the control run is the same initial conditions as the op but lower resolution.

That makes sense wrt uncertainty. I did know the Control uses the same IC.

[WinterWxLuvr]

I just wonder what perturbed actually entails.

[CAPE]

I just wonder what perturbed actually entails.

You deep thinker you. Likely some algorithm that alters the IC for each member.

[CAPE]

I used my friend Google and found this wrt GEFS:

 

The control member uses these initial conditions directly, while the perturbed members use an initial condition which has been obtained using an Ensemble Transform Bred Vector (ETBV) method that is designed to pick out the fastest growing modes in the model (with the hope that these simulate the fastest growing sources of uncertainty in the atmosphere). These perturbations are also subjected to stochastic physics perturbations.

 

https://www.ral.ucar.edu/guidance/guide/gfs/

[dtk]

I've always wondered what people were talking about with parametrization.  

 

Wikipedia tells me that it refers to processes that aren't explicitly resolved by a model and it gives some examples "....the descent rate of raindrops, convective clouds, simplifications of the atmospheric radiative transfer on the basis of atmospheric radiative transfer codes, and cloud microphysics." 

 

I'm wondering how different these parameters can be from model to model and how large a role they play in the differences we see between solutions.  

Thanks for starting this thread!  Very cool idea.

So it is exactly as you describe.  Convective parameterizations have huge impacts on NWP and are usually the easiest to describe.  Let's say we have a coarse model at something like 60 km resolution.  In reality, it takes 5 grid points to "resolve" (represent) the components of a wave.  So for practical purposes 30 km grid spacing means that you can really only explicitly resolve features that are less than 150 km. 

 

Now let's imagine a 30km grid box and in it we have scattered cumulus clouds.  We cannot resolve the clouds, but we know that the clouds will have impacts through containing saturated air, radiation, latent heating, and so on.  So, instead of "resolving" the cloud, we instead try to represent the effect of having clouds within that box in terms of the effects assumed.  

 

In grid boxes that have unstable profiles and assuming some lifting mechanism, the convective parameterization will attempt to represent the stabilizing effect of cumulus convection taking place, condensation, redistribution of heating, and eventually precipitation.  The precipitation processes are also greatly impacted by the microphysics (representation of the cloud, rain, graupel, ice particles) etc.

 

The convective schemes are typically divided into non-precipitating, shallow convection and deep convection.  The scheme used in the GFS (Simplified Arawaka-Schubert) is based on a quasi-equilibrium theory and is activated through moisture convergence and the availability of deep layer conditional instability.  The scheme used at the ECMWF is called Tiedtke and is based on a mass-flux approach.

 

The graphic on the ECMWF website is pretty good:

http://www.ecmwf.int/en/research/modelling-and-prediction/atmospheric-physics

 

You can see the many processes that are parameterized, such as radiation, diffusion, clouds, land processes, etc.

 

As an example, here is one case from September for the GFS divided into the total precip.:

 

and how much of it was the result of the convective scheme:

 

You can see how critically important the cumulus scheme is.

 

One last thing, these schemes really need to be "scale-aware" and change drastically as the horizontal resolution continues to increase.  The resolutions between 2-10km is generally considered to be a "grey zone" whereby it is actually quite tricky since deep convection is only partially resolved.

[dtk]

What an incredible thread. I had some questions a couple of weeks back about the ensembles. I was wondering what was different upon initialization to cause them to end at different conclusions. Or are they all run with exactly the same data and the differences are from chaos and slight tweaks throughout the run

Ensembles for NWP/medium range always use some combination of initial perturbations to represent analysis uncertainty and model perturbations to represent uncertainties in the models themselves (see conversation above regarding the need for parameterization of processes).

 

How the initial perturbations are created can vary from center to center, but ECMWF, NCEP, and Canada all use at least some information from an ensemble-based data assimilation scheme.  The ECMWF also supplements their initial perturbations with something called singular vectors, a concept that is likely familiar to those that have taken linear algebra.  In this case, the singular vectors are derived by trying to optimize large perturbation growth for a given lead time and norm.  In this case, I believe they either use 24 or 48 hours and use total energy.  Regardless, the point of initial perturbations is that they represent analysis error/uncertainty.  They are generally coherent in space and small amplitude perturbations.

 

Model "perturbations" come in a variety of forms.  In Canada, they effectively run a multi-model ensemble whereby some of their members actually use different parameterizations.  So one member would use Tiedtke instead of SAS (this is not what they actually use, but I am trying to tie in from the discussion above).

 

ECMWF and NCEP both use various forms of more stochastic parameterizations to represent model error.  You can read up on stochastic processes on Wikipedia (https://en.wikipedia.org/wiki/Stochastic_process).  You can think of this as being pseudo-random changes made on the fly within the model to the parts of the model that are thought to be uncertain.  Currently, the GEFS uses STTP: stochastic total tendency perturbations.  The model is literally perturbed with correlated noise in a particular way to represent model error.  ECMWF uses a combination of stochastic energy backscatter (SKEB) and stochastically perturbed physics tendencies (SPPT).  In as simple terms as I can think of:  SKEB is meant to inject energy back into the model at the truncation scale.  SPPT is like STTP but instead of perturbing the model tendencies through the integration, *only* the tendencies from the physics (cumulus, boundary layer, etc.) are perturbed.  Again, these are done in a coherent, correlated way as to try to represent the model error/uncertainty.

[dtk]

I used my friend Google and found this wrt GEFS:

 

The control member uses these initial conditions directly, while the perturbed members use an initial condition which has been obtained using an Ensemble Transform Bred Vector (ETBV) method that is designed to pick out the fastest growing modes in the model (with the hope that these simulate the fastest growing sources of uncertainty in the atmosphere). These perturbations are also subjected to stochastic physics perturbations.

 

https://www.ral.ucar.edu/guidance/guide/gfs/

Actually, with the last implementation the GEFS now uses a transformed set of perturbations derived from the EnKF that is part of the actual data assimilation.  They are no longer explicitly the "bred", fastest growing modes, but instead represent perturbations from analysis uncertainty.

[CAPE]

Thanks for the responses dtk. This is awesome stuff. Love when someone really knows the nuts and bolts on technical topics and can explain it well. Very impressive.

[dtk]

Yeah, great question about the ensembles. I see the word perturbed and would love to dtk explain that via an example.

I think that the initial perturbations are pretty straightforward to understand.  Just think if we started the GFS with initial conditions from the Canadian, Japanese, ECMWF, UKMO, etc., analyses....we'd get different answers for each realization.

 

For the model "perturbations", let's say we have a model equation that prescribes how the temperature will change at a grid point:

T(1) = T(0) + dT(dyn)*dt + dT(physics)*dt

 

Here, 0 is initial time, 1 is later time, dT is temperature tendency, dt is delta time.  So, the temperature at the grid point will change based on dynamics (advection, etc.) and physics (radiation, etc.).  Now, let's make a small modification to that equation

T(1) = T(0) + dT(dyn)*dt + dT(physics)*dt*Ran

 

Here, Ran is simply a random number drawn from a random pattern with prescribed, very small amplitude.

 

So if the control ends up staying the new temperature will be 25.0 degrees, perhaps the other members would be something like:

member 1: 24.9

member 2: 24.8

member 3: 25.1

member 4: 25.4

 

and so on.  The random numbers/pattern used for each member is unique to that member.  The amplitudes of the perturbations will generally be small for any variable/time step, but accumulate through time as the model progresses.

[dtk]

Thanks for the responses dtk. This is awesome stuff. Love when someone really knows the nuts and bolts on technical topics and can explain it well. Very impressive.

This is actually good practice for me.  I need to become much better at explaining technical topics to a non-technical audience.

[CAPE]

This is actually good practice for me.  I need to become much better at explaining technical topics to a non-technical audience.

Ha I know exactly what you mean. I teach industrial automation to folks with a diverse background....topics such as  feedback control, sensors, and calibration, and so I am quite familiar with measurement uncertainty analysis (RSS etc). Not everyone is receptive to differential equations, so you have to use other techniques to get the concepts across. Anyway, great stuff and much appreciated.

[Bob Chill]

This is actually good practice for me. I need to become much better at explaining technical topics to a non-technical audience.

It would be hard to find a better platform loaded with an interested but non-technical audience than amwx.

[WinterWxLuvr]

So interesting.

[stormtracker]

I always tried to get this mofo to speak about this at a conference.  This was my idea and I'm suing.

 

Settlement = dt*1.2(money)(dyn)+1(yuuuge)

[Ellinwood]

Awesome thread. Thought I would add a link to some other dtk NWP tidbits from threads past...

Comparing 0z/12z runs to 6z/18z runs:

http://www.americanwx.com/bb/index.php/topic/32149-gfs-12z-and-0z-runs-compared-to-6z-and-18z-runs/

[clskinsfan]

Thanks so much for this information dtk. If you are a tech geek like me. This is one of the greatest threads in Mid Atlantic forum history. And it helps to really understand the incredible amount of work that has gone into getting the models to the level they are today. It is truly a great human achievement.

[cae]

I agree with everyone else here.  This is a great thread.  Thanks dtk.

[usedtobe]

It's actually a good thread to have pinned. 

[stormtracker]

It's actually a good thread to have pinned.

Agreed. Alto this will definitely change the settlement formula.

Sent from my iPhone using Tapatalk

[Amped]

Nice writeup DTK. I find people only complain about models when their snow forecasts don't pan out.  

 

During the  9/29 rainstorm, I was progged to get 0.5" of rain on all models except the HRRR.  They ended up off by one decimal place and nobody here even mentioned this, it was a huge win for the HRRR and huge loss for every other model.