May 24 - July 30
Determining When and How a Random Forest Adds Value to Day 1 SPC Hail Forecasts
Shannon McCloskey, Eric D. Loken, David E. Jahn, Christopher Karstens, and Bryan Smith
What is already known:
- Random forests (RFs) produce skillful, reliable probabilistic guidance for next-day precipitation and severe weather and are starting to be used in real-time operations.
- Day 1 RF-based severe hail probabilities have been shown to be at least as good as corresponding day 1 Storm Prediction Center human forecasts, over many cases.
- Next-day RF and SPC hail probability forecasts can differ in magnitude and areal coverage. How often and why these differences occur is not currently known.
What this study adds:
- In most cases, the RF and SPC hail forecasts differ by no more than one SPC outlook category.
- Most frequently, RF probabilities fall one outlook category below that forecast by the SPC. Over many cases, the smaller RF probabilities are associated with lower observed report frequencies, suggesting the RF often successfully reduces false alarm compared to the SPC.
- When the RF forecasts at least one outlook category higher (lower) than the SPC, ensemble mean storm attribute variables (e.g., daily maximum 2-5km updraft helicity and upward and downward vertical velocity) tend to have higher (lower) absolute values, indicating stronger (weaker) simulated storms. Conversely, the distribution of the simulated storm attribute variables does not change much when the SPC forecasts at least one outlook category higher or lower than the RF.
- These findings suggest that RFs add value to the SPC by calibrating their probabilities based on the strength of simulated storms, while SPC forecasters add value to the RF by analyzing other (meteorological and non-meteorological) variables.
Abstract:
This study investigates when an RF algorithm’s day 1 severe hail probabilities differ from corresponding Storm Prediction Center (SPC) human-generated probabilities by at least one SPC outlook category. The goal of this study is to determine when an RF is most and least likely to add value to day 1 SPC human hail forecasts. RF forecasts are trained on forecast variables from the High-Resolution Ensemble Forecast System, version 2.1 (HREFv2.1) and observed SPC hail reports, using 627 days of data from May 2018 through April 2020. RF forecasts are compared against a continuous version of human-generated day 1 SPC hail forecasts, produced daily at 06z.
Analysis shows that the RF is especially skillful in reducing false alarm by forecasting one outlook category lower than that of the SPC. Additionally, when the RF forecasts at least one outlook category higher (lower) than the SPC, ensemble mean storm attribute variables including maximum 2-5 km updraft helicity, maximum upward vertical velocity, and maximum downward vertical velocity tend to have higher (lower) absolute values. Meanwhile, the distribution of these variables does not change much when the SPC forecasts at least one outlook category higher or lower than the RF. These findings suggest that RFs add value to the SPC by calibrating their probabilities based on the strength of simulated storms, while SPC forecasters add value to the RF by analyzing other (meteorological and non-meteorological) variables.