We perform separate log-linear regressions, for each numeric format, between computational performance and release date. The results from these regressions are used to estimate the average performance improvement relative to FP32. Confidence intervals for this difference are derived from the standard errors of the regression averages.

We assume that the aggregate performance trends across ML accelerators in our dataset are representative of hardware used for ML. This is an imperfect assumption, because a few leading hardware models are used more often. We believe that including other ML hardware is nevertheless useful to better estimate performance trends, but if flagship ML hardware follows different trends to ML hardware more broadly, these results will be less accurate.

For average performance improvement to reflect performance improvement at a given time, we must assume that the individual performance trends are close to parallel. So far, this appears to have held true.