Tornado warning false alarms and the optimum warning lead time

[AtticaFanatica]

There have been some great discussions over the past few months regarding the current state of the warning process in the context of the horrible tragedies in Joplin and Tuscaloosa. I wanted to draw your attention to a few articles that are about to be published in the coming months that I think the community here might find interesting.

The first topic is one discussed here often: tornado warning false alarms. A paper that will appear in Weather & Forecasting, Brotzge et al. (2011), summarizes a five year climatology of tornado false alarms. The abstract:

During 2008 approximately 75% of tornado warnings issued by the National Weather Service (NWS) were false alarms. This study investigates some of the climatological trends in the issuance of false alarms and highlights several factors that impact false alarm ratio (FAR) statistics. All tornadoes and tornado warnings issued across the continental U.S. between 2000 and 2004 were analyzed, and the data were sorted by hour of the day, month of the year, geographical region and weather forecast office (WFO), the number of tornadoes observed on a day in which a false alarm was issued, distance of the warned area from the nearest NWS radar, county population density and county area.

Analysis of the tornado false alarm data identified six specific trends. First, the FAR was highest during non-peak storm periods, such as during the night and during the winter and late summer. Second, the FAR was strongly tied to the number of tornadoes warned per day. Nearly one-third of all false alarms were issued on days when no tornadoes were confirmed within the WFO's county warning area. Third, the FAR varied with distance from radar, with significantly low estimates found beyond 150 km from radar. Fourth, the FAR varied with population density. For warnings within 50 km of a NWS radar, FAR increased with population density; however, for warnings beyond 150 km from radar, FAR decreased regardless of population density. Fifth, the FAR also varied as a function of county size. The FAR was generally highest for the smallest counties; the FAR was ~80% for all counties < 1000 km² regardless of distance from radar. Finally, the combined effects of distance from radar, population density, and county size led to significant variability across geographic regions.

This is available for early-online release, so I think anyone can access the pdf to read the study here.

Some key points:

1) The tornado false alarm rate (FAR) was highest in "off-peak" tornado times (overnight) and was lowest during peak tornado times.

2) FAR was much higher in the Midwest and Southeast than in the Plains and West regions.

3) FAR was negatively correlated with the number of tornadoes that were reported in a day.

4) FAR decreased beyond ranges of 150 km from radars. [No, I wasn't expecting that either.]

5) FAR was positively (negatively) correlated with population density (county size).

Regarding the regional differences, this hearkens back to the big discussion about James Spann's recent criticism of blanket QLCS tornado warnings. The authors here, however, argue the regional differences in FAR result from the size of the counties in the regions. This study uses data from 2000-2004, before the polygon warnings, so the scoring proceeds on a county basis (e.g., if a tornado warning encompassed two adjacent counties and the tornado only went through one county, it counts as one hit and one miss). The larger county sizes in the Plains and in the West helped to lower the FAR in those areas. They do not comment on the type of convection possibly playing a role.

The FAR decrease at far distances is a surprise as you'd expect with poorer radar coverage that the accurate detection of tornadoes would decrease. The authors argue that the lower FAR far from 88Ds results from fewer warnings being issued. It turns out that the probability of tornado detection also is decreased simply because fewer warnings are being issued at locations far from WSR-88Ds. With poor radar coverage, forecasters likely feel less comfortable issuing tornado warnings. These results should be on every CASA promotional medium.

The second topic is another one we've talked a lot about here: how much time is too much time for a warning. I've made mention a few times about how the AMS has introduced a new peer-reviewed journal entitled Weather, Climate, and Society that focuses on societal impacts of weather and climate. A study, Hoekstra et al. (2011), that will appear in that journal aims to find the optimum warning lead time based on what citizens themselves think.

Abstract:

Tornado warnings are currently issued an average of 13 minutes in advance of a tornado (Golden and Adams 2000) and are based on a warn-on-detection paradigm (Erickson and Brooks 2006). However, computer model improvements may allow for a new warning paradigm, warn-on-forecast, to be established in the future (Stensrud et al. 2009). This would mean that tornadowarnings could be issued one to two hours in advance, prior to storm initiation. In anticipation of the technological innovation, this study inquires whether the “warn-on-forecast” paradigm for tornado warnings may be preferred by the public (i.e., individuals and single families). Our sample is drawn from visitors to the National Weather Center in Norman, Oklahoma. During the summer and fall of 2009, surveys were distributed to 320 participants to assess their understanding and perception of weather risks and preferred tornado warning lead-time. Responses were analyzed according to several different parameters including age, region of residency, educational level, number of children, and prior tornado experience. A majority of the respondents answered many of the weather risk questions correctly. They seemed to be familiar with tornado seasons; however, they were unaware of the relative number of fatalities caused by tornadoes and several additional weather phenomena each year in the United States. The preferred lead-time was 34.3 minutes according to average survey responses. This suggests that while the general public may currently prefer a longer average lead-time than the present system offers, the preference does not extend to the one to two hour time-frame theoretically offered by the warn-on-forecast system. When asked what they would do if given a one-hour lead-time, respondents reported that taking shelter was a lesser priority than when given a 15-minute lead-time, and fleeing the area became a slightly more popular alternative. A majority of respondents also reported the situation would feel less life threatening if given a one-hour lead-time. These results suggest that how the public responds to longer lead times may be complex and situationally-dependent, and further study must be conducted to ascertain the users for whom the longer lead-times would carry the most value. These results form the basis of an informative stated-preference approach to predicting public response to long (> 1 hour) warning lead times, using public understanding of the risks posed by severe weather events to contextualize lead-time demand.

Again, you can read the whole pdf via early online release here.

Key points:

1) Respondents suggested that ~15 minutes was the minimum lead time to take shelter.

2) The ideal lead time was ~35 minutes and 40% of respondents said there could be too much lead time.

3) Fewer people stated they would take shelter with a 1 hour lead time than with 15 minutes of lead time.

4) More people would attempt to flee given the longer warning lead time.

5) Younger people preferred longer lead times while older people preferred shorter lead times.

This type of work has important implications for the warn-on forecast initiative if hour-long warning lead times do ever become a reality. Specifics about the make-up of the sample and other information can be found in the paper.

Both of these papers are short, easy reads, so I'd recommend them.

[kab2791]

Sure, you could "flee" your area with 1 hour lead time, but IMO that could lead to more problems due to the dangerous possibility of a rapid shift in strength/track of a tornadic supercell, excess traffic, or the possibility of a completely new storm to rapidly develop and pose a threat.

I guess it wasn't too surprising that increased distance from radar sites leads leads to a smaller FAR given the warnings wouldn't be issued unless there is a strong indication of a possible tornado/funnel cloud.

[AtticaFanatica]

Sure, you could "flee" your area with 1 hour lead time, but IMO that could lead to more problems due to the dangerous possibility of a rapid shift in strength/track of a tornadic supercell, excess traffic, or the possibility of a completely new storm to rapidly develop and pose a threat.

Yeah, I generally view that development as a bad thing.

I guess it wasn't too surprising that increased distance from radar sites leads leads to a smaller FAR given the warnings wouldn't be issued unless there is a strong indication of a possible tornado/funnel cloud.

Once I saw the explanation, it made sense, but that would not have been my first guess.

[Jonbo]

Part of the issue is that until a technology much improved from Radar is implemented, the FAR will be on the higher end. You can only do so much with radar but increased amateur and seasoned chasers has helped to confirm real tornadoes/threats.

The band-aid to get people into shelter seems to be Tornado Emergency because of criteria needed for that. Granted, rural areas will rarely have them but cycles back to the chasers on the ground. It's also bad because with so many amateur chasers out there now, new hazards come up.

Overall though, technology has to improve and be widely implemented.

[isohume]

4) FAR decreased beyond ranges of 150 km from radars. [No, I wasn't expecting that either.]

I can see how this is the case. You'll see the best rotation through a deeper and lower layer the closer a storm is to the radar. At long distances, the radar is sampling the elevated structure of the storm and therefore will not detect the llvl mesos and gate to gates. Thus, the close ranged storms will get warned on more often than the long ranged storms leading to a higher FAR closer in to the 88D.

I'd expect a lower POD and lower FAR at longer ranges.

[AtticaFanatica]

Do you guys think that researchers should be aiming for lead times that match what the public deems the best (let's say for argument, that's 35 minutes) or apply a "the longer, the better" approach (within reason, not like 6 hours)?

[Master of Disaster]

This has been an ongoing discussion in our office since Tuscaloosa and the discussion only intensified with Joplin. Unlike alot of other public warning systems, ours allows for replies so I get a pretty good idea of how the public is viewing what we send. I change the format on what I am sending pretty often to see how the warnings are perceived by the public. If I forward out a Flash Flood Warning or Tornado Warning with the standard NWS verbiage, it gets ignored or taken more of as an alert. If I lead with a headline like "Tornado confirmed" I receive a significant increase in registrations on the order of 500 or so as users forward the message to others and they then register. All total, some 115,000 residents are on the system as of today. We send ONLY life or property threatening information. I DO NOT send fog advisories, heat watches and warnings or any of the other non emergency type of warnings. As a result, the public knows when they receive an alert, its something significant EVERY time.

As far as lead time, I think 5 minutes is adequate IF its for a confirmed event. What we certainly cant do is what happened in Joplin where we hit the alarm, stop, and then retrigger the alarm after a significant delay.

If I had two pieces of advice for the NWS its to add a new Tornado Alert or something similar for events that are not verified. Tornado warning for events that are verified and Tornado Emergency for the same reason those are used now.

The other thing would be to add a "Extreme" designation to the SPC outlooks. I hate the use of "SLIGHT" and spend entirely to much time explaining how SLIGHT doesnt mean we will not see storms. By adding an "Extreme" designation, its my hope that it would GRAB attention but it could only be used for Extreme events like what happened with the Super Outbreak. They somewhat have the idea with PDS statements, but those get lumped in with normal watch verbiage so they get overlooked quite often.

[isohume]

If I had two pieces of advice for the NWS its to add a new Tornado Alert or something similar for events that are not verified. Tornado warning for events that are verified and Tornado Emergency for the same reason those are used now.

You don't want conflicting messages for the same type of serious storms. It is too confusing for the public and the wrong action is more likely to be taken. Say a "Tornado Alert" is issued for a doppler based tor (I'm assuming that's what you mean by "not verified") and the verbiage says something like..."THIS TORNADO HAS NOT BEEN VERIFIED...BUT IT HAS THE POTENTIAL". This makes it seem like it's not a serious or immediate threat, when in fact it very well could be. After a couple of these issued in their area and no tornado affected them...they will most likely not take cover the next time one is issued. Ideally, you want the public to take appropriate action every time upon hearing the unambiguous words "TORNADO WARNING". The public mindset needs to be more of..."oh a tornado warning, I need to take cover just to be sure"...instead of "oh a tornado alert, probably no big deal".

[Master of Disaster]

You don't want conflicting messages for the same type of serious storms. It is too confusing for the public and the wrong action is more likely to be taken. Say a "Tornado Alert" is issued for a doppler based tor (I'm assuming that's what you mean by "not verified") and the verbiage says something like..."THIS TORNADO HAS NOT BEEN VERIFIED...BUT IT HAS THE POTENTIAL". This makes it seem like it's not a serious or immediate threat, when in fact it very well could be. After a couple of these issued in their area and no tornado affected them...they will most likely not take cover the next time one is issued. Ideally, you want the public to take appropriate action every time upon hearing the unambiguous words "TORNADO WARNING". The public mindset needs to be more of..."oh a tornado warning, I need to take cover just to be sure"...instead of "oh a tornado alert, probably no big deal".

Good point, but thats what we do now and it doesnt seem to be working that well. Perhaps with the dual pol upgrades, the FAR might drop significantly.

[isohume]

Good point, but thats what we do now and it doesnt seem to be working that well. Perhaps with the dual pol upgrades, the FAR might drop significantly.

I don't see the FAR dropping because of Dual Pol. Tornado warnings will still be issued off the legacy base/RPG products like SRM and Z. However, I could see flash flood warning numbers improve since Dual Pol will aide precipitation processing by better identifying hail. The FAR for TORs will probably remain high until improved radar science is brought into operations.

[Master of Disaster]

I don't see the FAR dropping because of Dual Pol. Tornado warnings will still be issued off the legacy base/RPG products like SRM and Z. However, I could see flash flood warning numbers improve since Dual Pol will aide precipitation processing by better identifying hail. The FAR for TORs will probably remain high until improved radar science is brought into operations.

The correlation coefficient phases wouldn't improve the FAR for tornadoes at all you think?

[isohume]

The correlation coefficient phases wouldn't improve the FAR for tornadoes at all you think?

No, CC will identify differences in phases changes of the specific scatterer. Ie: non-meteorological scatterers like birds/insects have complex and changing returns in the horizontal and vertical and therefore a low CC...hail has somewhat complex differences, so a moderate CC...and scattering by snow/rain is well-behaved, so they will have a high CC.

CC will help you identify areas of different types of targets. It will not help with gate to gate rotation or identifying descending mesos, etc.

[Master of Disaster]

No, CC will identify differences in phases changes of the specific scatterer. Ie: non-meteorological scatterers like birds/insects have complex and changing returns in the horizontal and vertical and therefore a low CC...hail has somewhat complex differences, so a moderate CC...and scattering by snow/rain is well-behaved, so they will have a high CC.

CC will help you identify areas of different types of targets. It will not help with gate to gate rotation or identifying descending mesos, etc.

Understood, but wouldn't it give you an indication of debris within a vortex the same way? So if you compared RV and BR with it, you would have a pretty good indication that something is on the ground?

[isohume]

Understood, but wouldn't it give you an indication of debris within a vortex the same way? So if you compared RV and BR with it, you would have a pretty good indication that something is on the ground?

Yeah Dual Pol will highlight debris balls decently. However, debris balls are already easily identifiable through the legacy products. Dual Pol will have the same limitations as the legacy products wrt to debris balls. That being, debris balls will need to be relatively close to the radar in order to pick them up. In any case, a tornado warning does not count on or wait for a debris ball to be identified in order to be issued.