The negative binomial multiple changepoint algorithm for count data with allowance for over-dispersion

Abstract

Over-dispersion is common in discrete data arising from random count processes. Several count data models have been discussed in available literature, but modeling overdispersed data is still problematic in changepoint analysis. This study presents a hybrid likelihood-based algorithm that detects and produces reliable estimates of multiple change points for equally dispersed and over-dispersed data series; hence its advantage over other discrete changepoint techniques. The Negative Binomial Multiple Change Point algorithm is tested using synthetic data from the negative binomial distribution. Changes in the underlying data distribution are detected by conducting a likelihood ratio test on several partitions of data obtained through stepwise recursive binary segmentation, and maximum likelihood estimates of changepoints obtained. Critical values of the likelihood ratio test are developed and used to check for statistical significance of estimated change points. The simulation study further assesses the performance of the NBMCPA under varying sample sizes and locations of true changepoints. Various performance metrics are used to assess how well the model correctly identifies change points. Estimation errors are obtained as absolute deviations of known change points from the change points estimated under the algorithm. Algorithm robustness is evaluated through error analysis and visualization techniques including kernel density estimation and computation of metrics such as change point location accuracy, precision, sensitivity and false positive rate. The results indicate that the hybrid algorithm consistently detects change points that are present and does not erroneously detect changes when there are none. Changepoint location accuracy and precision of the algorithm increase with sample size, with best results for medium and large samples. Further model accuracy and precision are highest for changes located in the middle of the dataset compared to changes located in the periphery. The algorithm is found to work well for samples sizes with a threshold of 500, and where there are no temporal dependencies in the data. However, researchers who wish to model larger datasets are encouraged to develop critical values for the likelihood ratio test that would accommodate sample sizes larger than n=500. The algorithm is applicable to several real-word scenarios and is recommended for modeling data arising from count processes, provided that the data assumptions on dispersion and temporal dependence are upheld.