RUC GRIB's

[jgf]

I have been getting the RUC GRIB's from the links at ruc.noaa.gov

There is one file containing all of the model output fields, for each valid time, over the whole domain.

So, to get a single entire model run of 18 hrs, i have to download 18 files of about 30MB each.

I am only interested in a few of the output fields - sea level pressure and surface wind, mostly.., but also surface air temp, and maybe CAPE.

It would be great if I could get a single file containing just these fields, over the whole 18hr forecast.

It would be even greater if I could get them for a smaller geographic region, so that the file sizes are more manageable, as I often have somewhat restricted bandwidth when I need the files.

Is there any other source for RUC GRIB's?

[okie333]

I have been getting the RUC GRIB's from the links at ruc.noaa.gov

There is one file containing all of the model output fields, for each valid time, over the whole domain.

So, to get a single entire model run of 18 hrs, i have to download 18 files of about 30MB each.

I am only interested in a few of the output fields - sea level pressure and surface wind, mostly.., but also surface air temp, and maybe CAPE.

It would be great if I could get a single file containing just these fields, over the whole 18hr forecast.

It would be even greater if I could get them for a smaller geographic region, so that the file sizes are more manageable, as I often have somewhat restricted bandwidth when I need the files.

Is there any other source for RUC GRIB's?

http://nomads.ncep.noaa.gov/txt_descriptions/fast_downloading_grib.shtml

[wxjoe]

The best place to get the RUC grids (or any other NCEP model) is probably the real-time NOMADS site: http://nomads.ncep.noaa.gov/ - This server is supported 24x7 for real-time usage. I'm not aware of a more reliable (free) server from NOAA/NCEP.

How do you use the data? If you work with GrADS, you may be interested in taking advantage of the OpenDAP/GrADS-DODS server. See the GrADS-DODS user guide for information on how to use it and this paper for background info: ftp://iges.org/grads/gds/joew.AMS.2001.pdf

NCEP's NOMADS site does have a grib filter option which allow you to download only the data you need (eg. only certain variables, as you asked). Just click on the grib filter link next to the RUC entry on the table on the NOMADS home page.

Finally, you can take advantage of the "fast downloading" technique as described by okie333's link above.

Let me know if you have any questions. I'm far from an expert in this stuff, but I can do my best to lend a hand.

[jgf]

Thanks for the suggestions.

I tried the filter first, because it was easy, but I am only getting empty files. I'll continue to play with it.

It seems that using the filter, I will still need to download each valid time individually.

If i can get the "fast downloading" method to work, it seems like it can be pretty well automated.

One thing that isn't clear, is whether data read from each of the VT files can be written to one output file. in other words, can I get a single GRIB file containing the data for each VT?

It's also not clear whether, using this method, I can get the RUC for a restricted geographical area.

I'm not using GRADS for analysis of the data, but if it will make it easy for me to get the data, then I might use it just for that.

[wxjoe]

I tried the filter first, because it was easy, but I am only getting empty files. I'll continue to play with it.

It seems that using the filter, I will still need to download each valid time individually.

If i can get the "fast downloading" method to work, it seems like it can be pretty well automated.

One thing that isn't clear, is whether data read from each of the VT files can be written to one output file. in other words, can I get a single GRIB file containing the data for each VT?

Hmm... that's odd that you're getting empty files. If you continue to have problems you may want to contact Wesley (see email address at bottom of grib filter page) for help.

While you'll need to download a separate file for each projection, you can 'cat' them together. Forgive me if you know this, but just to be clear, I mean that you can do the following to condense all of the grib2 files into 1 file for an entire cycle: say all of your ruc files are RUC_2011-10-15_03Z_HTTz.gb2, where TT= the projection hour. You can run the following command, then, to condense all of the files together:

cat RUC_2011-10-15_03Z_H*z.gb2  > RUC_2011-10-15.gb2

This will only work on a platform that has cat - eg. a linux/unix platform (including mac os x). There are other ways to do this if you're working in Windows, such as using the CoreUtils package from the GnuWin32 project.

It's also not clear whether, using this method, I can get the RUC for a restricted geographical area.

Yeah, using the grib filter is going to be your best bet as far as restricting your download to a certain geographical area.

What are you using this data for, if you don't mind me asking? I'm always curious how people find cool and interesting ways to use this data.

[jgf]

Thanks.

I was never much of a unix geek, but at one time I knew more than I know now.

I'm doing this on windows.

I hesitate to mention my use, because no one will take me seriously...

I do navigation and tactics on racing sailboats; both ocean racing, and coastal/inshore racing.

I am looking for the best short term wind forecasts I can get. I currently use the GFS and COAMPS, as well as a few higher resolution subscription services - one is based on RAMS, and one is (as far as I can tell) a sort of hi-res version of GFS.

GFS is ok for long term (2-5 days) forecasts, and for tracking big systems but I need something with better temporal and spatial resolution.

I am thinking that RUC, and maybe HRRR might be better than my subscription services, and they are free.

As far as I can tell, HRRR gribs are not publicly available yet, but will be at some time.

[dtk]

Thanks.

I was never much of a unix geek, but at one time I knew more than I know now.

I'm doing this on windows.

I hesitate to mention my use, because no one will take me seriously...

I do navigation and tactics on racing sailboats; both ocean racing, and coastal/inshore racing.

I am looking for the best short term wind forecasts I can get. I currently use the GFS and COAMPS, as well as a few higher resolution subscription services - one is based on RAMS, and one is (as far as I can tell) a sort of hi-res version of GFS.

GFS is ok for long term (2-5 days) forecasts, and for tracking big systems but I need something with better temporal and spatial resolution.

I am thinking that RUC, and maybe HRRR might be better than my subscription services, and they are free.

As far as I can tell, HRRR gribs are not publicly available yet, but will be at some time.

The HRRR is a prototype system, not developed, maintained, nor run by NCEP folks...so you're only hope would be to get the grib files from NOAA/ESRL. It is still quite a ways out from being viable for implementation into the NCEP production suite.

The RUC is about to get a major overhaul in the coming months. NWS TIN available HERE, description/ESRL information HERE, some of the NCEP/EMC pre-implementation testing HERE, and access to some of the parallel files HERE.

[InstantWeatherMaps]

Thanks.

I was never much of a unix geek, but at one time I knew more than I know now.

I'm doing this on windows.

I hesitate to mention my use, because no one will take me seriously...

I do navigation and tactics on racing sailboats; both ocean racing, and coastal/inshore racing.

I am looking for the best short term wind forecasts I can get. I currently use the GFS and COAMPS, as well as a few higher resolution subscription services - one is based on RAMS, and one is (as far as I can tell) a sort of hi-res version of GFS.

GFS is ok for long term (2-5 days) forecasts, and for tracking big systems but I need something with better temporal and spatial resolution.

I am thinking that RUC, and maybe HRRR might be better than my subscription services, and they are free.

As far as I can tell, HRRR gribs are not publicly available yet, but will be at some time.

They said in an email to me that GRIBs should be bail able in 2012.

[dtk]

They said in an email to me that GRIBs should be bail able in 2012.

You mean the HRRR? I assume this is from ESRL somehow? Given that the RR won't even be implemented for several months, there is no way the HRRR is going to be implemented next year.

[InstantWeatherMaps]

You mean the HRRR? I assume this is from ESRL somehow? Given that the RR won't even be implemented for several months, there is no way the HRRR is going to be implemented next year.

Actually it was from someone at the NCEP EMC. And "bail able" should have said "be available"; I was autocorrected.

[wxjoe]

Thanks.

I was never much of a unix geek, but at one time I knew more than I know now.

I'm doing this on windows.

Oh, no problem! If you're interested in putting together the files like I stated, you can either use the utility I linked to or take advantage of the MS-DOS 'copy' command which apparently can be used like cat on linux/unix. (YMMV)

I hesitate to mention my use, because no one will take me seriously...

I do navigation and tactics on racing sailboats; both ocean racing, and coastal/inshore racing.

I am looking for the best short term wind forecasts I can get. I currently use the GFS and COAMPS, as well as a few higher resolution subscription services - one is based on RAMS, and one is (as far as I can tell) a sort of hi-res version of GFS.

GFS is ok for long term (2-5 days) forecasts, and for tracking big systems but I need something with better temporal and spatial resolution.

I am thinking that RUC, and maybe HRRR might be better than my subscription services, and they are free.

As far as I can tell, HRRR gribs are not publicly available yet, but will be at some time.

Cool! That sounds interesting.

The RUC/Rapid Refresh sounds like the right solution for you. As dtk mentioned, you can download the *parallel Rapid Refresh grids online from the link he gave, or just use the (likely more reliable) RUC grids. The Rapid Refresh ("RAP") will replace the RUC within the next few months (again, as dtk mentioned), though I wouldn't worry too much about this model change: nearly all of the same variables in the grib2 will continue to be produced (with a few exceptions, of course).

The HRRR grib2 grids are not publicly available just yet nor are the chances great that the model will be implemented officially at NCEP anytime soon. (The grib2 does exist, of course, it's just not being advertised for users outside of NOAA/NWS just yet.) The HRRR requires some massive computing power to say the least... though its unreliability in real-time availability and its constant tinkering do not make it a suitable model for operational weather forecasting, it seems many forecasters are nonetheless using it as part of their forecasting/nowcasting routine.

*In case you didn't know: Running a model in parallel is a term used to designate a model that is running on the supercomputers at NCEP in parallel with the operational model so that the upcoming model can be thoroughly tested before becoming operational. When a model is run in parallel it should not expected to be available 24x7 and should not yet be used operationally.

[jgf]

I did get the filter to work - i think i was not associating the correct variable with the layer i specified, resulting in empty files.

For wind, I am just choosing 10m, and "all" variables.

For the whole model domain, I am getting files of about 125kb, still too big for restricted bandwith at sea (iridium sat phone), but ok for when I am in cellular data range.

If I restrict the download to a typical area of interest, I am getting <25kb, which is ok, even for Iridium, as long as I stay below 5 or so time steps.

I have some questions about the model itself, but I think I will start another thread, and confine this thread to accessing the data.

one question that might be appropriate for this thread concerns plotting the data.

When I use the "extract subregion" option, I get bounds to the plots that are not parallel to lat/long, even though that's how the subregion is specified.

the RUC grib is in a lambert conformable projection, and I want to be sure there is no problem with my plotting software handling this projection.

Because I have seen the issue with two separate plotting programs, I suspect the issue is with the data extraction.

Does anybody have any ideas about this?

I have attached two plots (done in the Panoply data viewer). One is equirectangular, and one is lambert conformable.

[jgf]

I communicated with the author of Panoply, and he confirmed that the issue is with data extraction from the RUC grib.

Apparently, the data extraction routine takes a rectangular subset of data from the lambert conformable RUC grib, with lat/long boundaries that approximately match those entered in the dialog boxes.

if one plots the extracted data withe the same lambert conformable projection parameters as used in the extracted data set, the Panoply plot looks fine. In my plot above, I didn't know what the parameters were.