Cory Posted January 12, 2011 Share Posted January 12, 2011 Use GrADS instead. It's much faster. You shouldn't be taking any more than 1/2 second per map for generation (assuming you already have made GRIB files for GrADS to read), and that's even pretty slow (I can generate 5 or so 800x600 maps per second from 0.5 deg GFS on a VirtualBox Linux server running on a single core of an ordinary laptop... decrease image size to 400x300 and it's around 20 per second). That may be what ends up happening. I have GrADS, but I like NCL's graphics better and wanted to use them for this site since it helps make it different from others. The problem is, I have to convert NCL images from postscript using Imagemagick commands built into the scripts. Imagemagick (using every variety of options I can find) can't seem to get the image file size small enough while maintaining quality, plus it takes way too long to convert. Link to comment Share on other sites More sharing options...
okie333 Posted January 12, 2011 Share Posted January 12, 2011 That may be what ends up happening. I have GrADS, but I like NCL's graphics better and wanted to use them for this site since it helps make it different from others. The problem is, I have to convert NCL images from postscript using Imagemagick commands built into the scripts. Imagemagick (using every variety of options I can find) can't seem to get the image file size small enough while maintaining quality, plus it takes way too long to convert. Exactly. And how long does it take to do each map? Based on what you said it's around 30 seconds... definitely way too long. For GrADS: Gif is 30% faster than png, but the files are 2x bigger on average. Black contour-only files work super-fast, leading me to believe that image compression (or possibly pixel fill rate, but there are reasons that I think this isn't the case) is the bottleneck. If you don't like the GrADS graphics, try printing to an image twice the size in each dimension and shrinking it using ImageMagick... I believe shrinking an image is a fairly fast operation if you use linear scaling... certainly should be able to do 2 seconds per image if you generate a 1600x1200 PNG (gif doesn't work with this method) then downscale to 800x600. Be sure to use thicker settings for the maps and text. Link to comment Share on other sites More sharing options...
Amped Posted January 14, 2011 Share Posted January 14, 2011 What is the difference between this and the NCEP WRF? Link to comment Share on other sites More sharing options...
Cory Posted January 14, 2011 Share Posted January 14, 2011 What is the difference between this and the NCEP WRF? Different horizontal resolution, physics, vertical levels, and different initialization and boundary conditions. The NCEP WRF-ARW and ours both use the same dynamics solver (ARW) but differ otherwise in several ways. Link to comment Share on other sites More sharing options...
Cheeznado Posted January 15, 2011 Share Posted January 15, 2011 I really hate to complain since it was great of you guys to do this, but how much past 0 and 12Z is this run? Right now it is approaching 06Z and the 00Z run is not available yet. Link to comment Share on other sites More sharing options...
Cory Posted January 15, 2011 Share Posted January 15, 2011 I really hate to complain since it was great of you guys to do this, but how much past 0 and 12Z is this run? Right now it is approaching 06Z and the 00Z run is not available yet. Right now it takes about 6 hours after 00Z/12Z to see the images online (we're essentially presenting 06Z/18Z runs). It's initialized from GFS, so it takes almost 4 hours after synoptic time to even get the data for the WRF to run. It then takes about 1 hour 10 minutes to run the model itself, and another hour to post-process and upload. The post-processing is going to be changed though and that will speed things up by about an hour. As mentioned earlier, this is really a "beta version" because we wanted to get something out for everyone to use and make improvements and upgrades along the way (why am I sounding like an admin? ) Link to comment Share on other sites More sharing options...
okie333 Posted January 15, 2011 Share Posted January 15, 2011 Right now it takes about 6 hours after 00Z/12Z to see the images online (we're essentially presenting 06Z/18Z runs). It's initialized from GFS, so it takes almost 4 hours after synoptic time to even get the data for the WRF to run. It then takes about 1 hour 10 minutes to run the model itself, and another hour to post-process and upload. The post-processing is going to be changed though and that will speed things up by about an hour. As mentioned earlier, this is really a "beta version" because we wanted to get something out for everyone to use and make improvements and upgrades along the way (why am I sounding like an admin? ) So you took my idea about using GrADS? Link to comment Share on other sites More sharing options...
baroclinic_instability Posted January 15, 2011 Share Posted January 15, 2011 So you took my idea about using GrADS? I have no problem with the Grads idea either--it has plenty of configurability to make it look just as good as anything else. Link to comment Share on other sites More sharing options...
salbers Posted January 16, 2011 Share Posted January 16, 2011 That may be what ends up happening. I have GrADS, but I like NCL's graphics better and wanted to use them for this site since it helps make it different from others. The problem is, I have to convert NCL images from postscript using Imagemagick commands built into the scripts. Imagemagick (using every variety of options I can find) can't seem to get the image file size small enough while maintaining quality, plus it takes way too long to convert. I can understand processing postscript with NCL might be a bit slow. I'm still using the old NCAR graphics stuff (within NCL). The strategy here is using 'ctrans' to convert a gmeta file into a raster image ".x" file. Then Imagemagick can be used to quickly make a PNG. This is all used in our LAPS "on-the-fly" page. http://laps.noaa.gov...st/nph-laps.cgi As an idea the LAPS/WRF system could be used to post-process your WRF output. There are scripts contained therein to produce GIF images of the forecast fields, as can seen here: http://laps.noaa.gov/forecasts/ It works by running LFMPOST to convert the WRF netCDF into LAPS "FUA" and "FSF" files, then the NCL/NCAR graphics stuff can be run. Steve Link to comment Share on other sites More sharing options...
jcwxguy Posted January 19, 2011 Share Posted January 19, 2011 Looks good so far, but Will you guys be switching the scale on the total precip map? ( currently 0-.50 ) to something simular to a .01-.10 , .10-.25, .25-.50, ect? ( i know still in beta) Link to comment Share on other sites More sharing options...
Cory Posted January 24, 2011 Share Posted January 24, 2011 Today's 12z WRF run will not be available because I'm doing some upgrades/testing that I didn't have done in time for the run. I may have the cluster running for the 00z and be able to upgrade to the higher-res domain by then. I can't make any promises, but we'll see. Either way,there will be a 00z run today. Link to comment Share on other sites More sharing options...
Cory Posted January 27, 2011 Share Posted January 27, 2011 Anyone have feedback on the model performance itself? It's been running through two eastern snow storms and I'm wondering if anyone has any kudos, complaints, or grievances. Model physics and dynamics can be tweaked if we're seeing problems. Link to comment Share on other sites More sharing options...
LVblizzard Posted January 27, 2011 Share Posted January 27, 2011 Anyone have feedback on the model performance itself? It's been running through two eastern snow storms and I'm wondering if anyone has any kudos, complaints, or grievances. Model physics and dynamics can be tweaked if we're seeing problems. It performed very well with the storm 2 weeks ago, as well as with the one a week ago, especially with regards to the strength of the low pressure system. For yesterday's storm it had the precip shield slightly too far south, but then again most hi-res mesoscale models did. Link to comment Share on other sites More sharing options...
Cory Posted February 3, 2011 Share Posted February 3, 2011 Bad news--we've had a hard drive failure on the Beast computer and the WRF is down. I just noticed it when I got up to go to work last night, so I haven't had time to look into it yet. Needless to say, no 12Z run today (I know all 5 of you were waiting in suspense for it). Link to comment Share on other sites More sharing options...
baroclinic_instability Posted February 3, 2011 Share Posted February 3, 2011 Bad news--we've had a hard drive failure on the Beast computer and the WRF is down. I just noticed it when I got up to go to work last night, so I haven't had time to look into it yet. Needless to say, no 12Z run today (I know all 5 of you were waiting in suspense for it). I do use it and analyze it--so count me in as one of the 5. Link to comment Share on other sites More sharing options...
Cory Posted February 3, 2011 Share Posted February 3, 2011 Okay folks---FOLKS! Just try to remain calm, I'll get this going as fast as possible. I'm going to have to reinstall everything on a different drive and get it set up again--likely by tomorrow. In the meantime, enjoy whatever it is you people do. Link to comment Share on other sites More sharing options...
eekuasepinniW Posted February 3, 2011 Share Posted February 3, 2011 Anyone have feedback on the model performance itself? It's been running through two eastern snow storms and I'm wondering if anyone has any kudos, complaints, or grievances. Model physics and dynamics can be tweaked if we're seeing problems. I've noticed when I check the total snowfall/precip maps that there is a tiny little hole just to my east and another over by Sebago lake in Maine. It's been there for more than one storm. I thought at first it was picking up on terrain or something, but the Sebago lake area is flat as Kansas. I forgot to save the image showing it, sadly. Link to comment Share on other sites More sharing options...
Cory Posted February 3, 2011 Share Posted February 3, 2011 I've noticed when I check the total snowfall/precip maps that there is a tiny little hole just to my east and another over by Sebago lake in Maine. It's been there for more than one storm. I thought at first it was picking up on terrain or something, but the Sebago lake area is flat as Kansas. I forgot to save the image showing it, sadly. Is there a lake or something to your east in that area with the tiny hole? It knows where bodies of water are and won't accumulate snow on them and also doesn't count precip over them, so that might be the case. Link to comment Share on other sites More sharing options...
eekuasepinniW Posted February 3, 2011 Share Posted February 3, 2011 Is there a lake or something to your east in that area with the tiny hole? It knows where bodies of water are and won't accumulate snow on them and also doesn't count precip over them, so that might be the case. lol.. well that would explain that. The spot to my east must be Lake Winnipesaukee, just not exactly where it should be. Link to comment Share on other sites More sharing options...
Mallow Posted February 3, 2011 Share Posted February 3, 2011 Bad news--we've had a hard drive failure on the Beast computer and the WRF is down. I just noticed it when I got up to go to work last night, so I haven't had time to look into it yet. Needless to say, no 12Z run today (I know all 5 of you were waiting in suspense for it). I would use it, but it doesn't cover the west yet. Link to comment Share on other sites More sharing options...
Cory Posted February 4, 2011 Share Posted February 4, 2011 I would use it, but it doesn't cover the west yet. I'm workin' on it, I swear. Link to comment Share on other sites More sharing options...
Cory Posted February 4, 2011 Share Posted February 4, 2011 Okay we're back in business and will have a 00Z run today. Link to comment Share on other sites More sharing options...
baroclinic_instability Posted February 4, 2011 Share Posted February 4, 2011 Okay we're back in business and will have a 00Z run today. Any chance the images can be larger? Link to comment Share on other sites More sharing options...
Cory Posted February 4, 2011 Share Posted February 4, 2011 Any chance the images can be larger? I can make them larger but the file sizes get a bit unwieldy using NCL. I'll mess around with some settings today and see what I can do--I agree that they would be better if they were bigger. Link to comment Share on other sites More sharing options...
Cory Posted February 5, 2011 Share Posted February 5, 2011 Any chance the images can be larger? Should see larger 00Z images starting to show up in the next 30 minutes or so. The previous dimensions were 900x900 and the new ones are 1200x1200, which I think will help a lot. Some of the scripts are still acting goofy, such as temperature scales jumping around as you loop through them, but I'm tackling that tomorrow. I miraculously ended up with 4 days off work and finally have some time to dig into this stuff and fix some issues. Also we'll be extending the run time out to 36 hours starting tomorrow. Link to comment Share on other sites More sharing options...
baroclinic_instability Posted February 5, 2011 Share Posted February 5, 2011 Should see larger 00Z images starting to show up in the next 30 minutes or so. The previous dimensions were 900x900 and the new ones are 1200x1200, which I think will help a lot. Some of the scripts are still acting goofy, such as temperature scales jumping around as you loop through them, but I'm tackling that tomorrow. I miraculously ended up with 4 days off work and finally have some time to dig into this stuff and fix some issues. Also we'll be extending the run time out to 36 hours starting tomorrow. Nice! I will check out tonites 0Z run when it comes out. Link to comment Share on other sites More sharing options...
Cory Posted February 5, 2011 Share Posted February 5, 2011 Nice! I will check out tonites 0Z run when it comes out. Having some problems getting them all on the site in decent time tonight, but here's the new size for example vs old size: Link to comment Share on other sites More sharing options...
baroclinic_instability Posted February 5, 2011 Share Posted February 5, 2011 Having some problems getting them all on the site in decent time tonight, but here's the new size for example vs old size: Thanks--that is definitely a lot better. Also--is LAPS working yet? I am guessing this must still be a cold start--it seems to take a while for the precip fields to spin up. Link to comment Share on other sites More sharing options...
Cory Posted February 5, 2011 Share Posted February 5, 2011 Thanks--that is definitely a lot better. Also--is LAPS working yet? I am guessing this must still be a cold start--it seems to take a while for the precip fields to spin up. Yeah still on a cold start for now. I hope to get working on LAPS soon....but also have a lot of other personal stuff to get wrapped up before March. Link to comment Share on other sites More sharing options...
baroclinic_instability Posted February 5, 2011 Share Posted February 5, 2011 Yeah still on a cold start for now. I hope to get working on LAPS soon....but also have a lot of other personal stuff to get wrapped up before March. I definitely understand. It will definitely be interesting using it more once the kinks are worked out. I did use it for the SNE Noreaster bomb and it actually did a fair job of developing the surface low and bombing it out right on the eastern side of LI--better than most other high res and/or non-hydrostatic guidance except maybe the SUNY MM5 which did ok too--although to be fair all models stunk it up overall--but no model is made to accurately simulate the complex mesoscale interactions in that storm. Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.