Run Charts

Alan Picarillo, MD, FAAP

Medical Director of NICU/CCN

The Barbara Bush Children’s Hospital

Learning objectives:

  1. Describe why dynamic data Is preferable to static data for quality improvement
  2. Explain the basic components of a run chart
  3. Identify the basic rules of a run chart

Measurement of data is a core concept of quality improvement. Analysis of those collected data requires a distinct approach as compared to other areas of research. The usual research model for data is pre- versus post- intervention and that model is an incredibly important tenet of scientific inquiry. Many statistical tools are based upon those comparisons of data and outcomes.  The Model for Improvement, which is familiar to readers of this series, requires that one identifies measures to evaluate the impact of planned changes before considering the change ideas themselves.  The central role of measurement dates back to the work of Walter Shewhart and W. Edwards Deming and their effort to understand and measure data variation.

Traditional healthcare data are primarily static. Basic statistics are oriented towards cause and effect relationships in order to determine the significance of differential outcomes.  However, data for quality improvement is inherently time-oriented and therefore should be examined over time.   Aggregated data before and after an intervention can fail to show important trends which may be visible with more frequent or granular data.  Performance measured annually may not show changes which accurately reflect the impact of intervention or show improvement opportunities, such as monthly data might.  It is this display of dynamic data over time that forms the foundation for statistical data analysis in quality improvement.

A run chart is the most basic and commonly used graph of time-series data, but allows for more rigorous interpretation than simple linear graphs of data.  The x-axis represents time and is often plotted based on a specific measure of time (e.g., day, week, month) and the y-axis is the measure of interest.  Also included on a run chart is the centerline, or measurement of central tendency, which is typically the median FIGURE 1.  It is this centerline that allows for analysis of data variation and there are rules to determine significant data variations (signals) versus normal data variations (noise).  Also, annotations to the run chart and specific goal lines allow for a complete pictorial representation to illustrate the QI project for others.

Figure 1 Annotated run chart

Commonly used rules for detecting data signals in a run chart are the following FIGURE 2:

  1. Shift: Six or more consecutive points either above or below the median (centerline). Those values that fall on the median are skipped and do not add to or break a shift.
  2. Trend: Five of more consecutive points all going up or going down
  3. Too few or too many runs: A significant data variation can be signaled by either too many or too few runs, or crossings of the centerline. Critical value tables exist in the literature to determine significant variation (5% risk of failing the run test for random patterns of data)
  4. Astronomical data point: A data point that is unusually different from the rest of the data points.

Figure 2 Run chart rules

References

Langley GJ, Moen RD, Nolan KM, Nolan TW, Norman CL, Provost LP. The improvement guide: a practical approach to enhancing organizational performance. San Francisco, CA: John Wiley & Sons; 2009

Institute for Healthcare Improvement. Institute for Healthcare Improvement: science of improvement: establishing measures. http://www.ihi.org:80/resources/Pages/HowtoImprove/ScienceofImprovementEstablishingMeasures.aspx.  Accessed 21 Apr 2021

Perla RJ, Provost LP, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011;20:46 51.

Want to earn Continuing Education Credits? Click here

Download PDF