Investigating the Efficacy of Ensemble Sensitivity-Based Tools for Forecast Improvement in the NOAA Warn-on-Forecast System
Ensemble sensitivity analysis (ESA) is a computationally inexpensive tool that identifies dynamics relevant to meteorological outcomes by using a linear regression to relate the forecast to the ensemble state. This technique has been the subject of increasing study in recent times due to its utility in operational forecasting and research applications. Because of its linear assumption, ESA has been most widely used at large atmospheric scales which can typically be acceptably described through such processes. However, study has recently extended to ESA’s potential in short-term mesoscale applications that are often believed highly chaotic like convective processes, and results have proven generally promising that the technique can still find success with these phenomena. Left largely unaddressed is the potential for ESA utility with very small convective scales like those for individual storms in the NOAA Warn-on-Forecast System (WoFS). Because WoFS aims to serve as the first operational ensemble paradigm for short-term severe storm forecasts, this technique could theoretically prove useful to such forecasts in an eventual effort to improve warning lead times. Further, recent work has studied ensemble subsetting, a novel ESA-based tool which removes ensemble members with large errors in sensitive regions to form a smaller, theoretically more accurate ensemble subsets. If ensemble sensitivity is useful in WoFS, it is plausible that this method yields potential for automated forecast improvement of short-term convective signals in WoFS. This study diagnoses the utility of ESA and its derivative tools in WoFS. Using a 36-member WoFS-imitating ensemble, sensitivity fields are calculated for several variables and analyzed qualitatively for 4 supercell cases in May 2019 to determine if ESA is a useful tool in WoFS and how the method is affected by its specific configuration. The ability of ensemble subsetting to yield forecast benefit was then tested via two sets of experiments. The first attempts subsetting in an idealized setting in which a “truth member” is removed and compared to the subsets, serving as a theoretical upper limit to the utility of subsetting. A practical subsetting experiment was then performed using near-storm observations from the Targeted Observations by Radars and UAS of Supercells (TORUS) project. Results suggest that ESA does prove useful in WoFS, and its efficacy is systematically affected by its multiphysics nature such that state spread is stratified by boundary layer parameterization scheme. Idealized subsetting experiments show promise depending on the case analyzed, with distance thresholding permutations proving to provide the most forecast benefit. Practical subsetting results were mixed and difficult to trust given significant potential for errors in observations and methods. These analyses suggest that, despite challenges, ESA and its subsetting technique may represent worthwhile pursuits in endeavors to improve WoFS skill.