The Core Mathematics
Standard Bayesian conjugate models assume a stationary environment (the underlying data never changes). While mathematically optimal, they accumulate so much historical precision that they become completely rigid.
To track non-stationary data, we introduce a forgetting factor \( f \) (where \( 0 < f \le 1 \)). Before adding new data, we multiply our historical parameters by \( f \). This artificially inflates our uncertainty, allowing the model to "forget" outdated data and pivot to new trends.
We discount the shape parameters of the Gamma/Beta distributions:
\[ \alpha_{\text{new}} = f \cdot \alpha_{\text{prior}} + \text{new data} \] \[ \beta_{\text{new}} = f \cdot \beta_{\text{prior}} + \text{new data} \]We discount the Precision Matrix \( \Lambda = \Sigma^{-1} \). This naturally widens the confidence intervals before the new observation is incorporated:
\[ \Lambda_{\text{new}} = f \cdot \Lambda_{\text{prior}} + \Lambda_{\text{data}} \] \[ \boldsymbol{\mu}_{\text{new}} = \Lambda_{\text{new}}^{-1} (f \cdot \Lambda_{\text{prior}} \boldsymbol{\mu}_{\text{prior}} + \Lambda_{\text{data}} \mathbf{x}) \]We discount the entire \( \boldsymbol{\alpha} \) vector to allow for shifting user preferences:
\[ \alpha_{i, \text{new}} = f \cdot \alpha_{i, \text{prior}} + x_i \]By sacrificing pure Bayesian optimality, we gain the agility to track moving targets, changing click-through rates, and dynamic server loads—the exact techniques used in modern Kalman Filters and Multi-Armed Bandits.
Real-World Applications
Each model in this series is tailored to a specific type of data. Here is how they are applied in the real world:
Measures: Event counts in a fixed timeframe.
Use: Telecommunications (calls per minute), web analytics (server requests per second), and retail (customers entering a store per hour).
Measures: Success rates (proportions).
Use: The gold standard for A/B testing. Used to model click-through rates (CTR) on ads, email open rates, and medical treatment success probabilities.
Measures: Discrete time/steps (Failures before the 1st success).
Use: Modeling user patience. Examples include clicks before abandoning a webpage, products inspected before finding a defect, or number of throws missed before scoring.
Measures: Continuous time (Time between events).
Use: Reliability engineering and system monitoring. Used to measure the time until a server crashes, the lifespan of a lightbulb, or the time between API requests.
Measures: Continuous measurements & means.
Use: Industrial quality control (average temperature of a reactor), finance (average stock price), and healthcare (average heart rate of a patient).
Measures: 2D spatial coordinates (X, Y).
Use: Tracking moving objects in 2D space. Used in GPS navigation, radar systems tracking aircraft, and self-driving cars tracking surrounding vehicles.
Measures: Multi-category success rates.
Use: Used whenever there are more than two outcomes. Powers Natural Language Processing (topic modeling), email sorting (Spam, Promotions, Primary), and UI/UX tracking.
Measures: Discrete time/steps (Failures before \(r\) successes).
Use: Used when a process must hit a specific target. Modeling accident-free months before a driver files 3 claims, or users seeing an ad before achieving 5 conversions. Negative Binomial is similar to Geometric Model, where the number of trial is not fixed, but instead of a single success, there is a specific number of successes that may occur in any order, hence the use of binomial coefficient. The negative binomial distribution gives you the probability of achieving a fixed number of successes (\(r\)) on a specific total number of trials (\(x\)) (or alternatively, having a specific number of failures before that \(r\)-th success). Wikipedia