The Core Mathematics

Standard Bayesian conjugate models assume a stationary environment (the underlying data never changes). While mathematically optimal, they accumulate so much historical precision that they become completely rigid.

To track non-stationary data, we introduce a forgetting factor \( f \) (where \( 0 < f \le 1 \)). Before adding new data, we multiply our historical parameters by \( f \). This artificially inflates our uncertainty, allowing the model to "forget" outdated data and pivot to new trends.

Counts & Probabilities (Poisson, Binomial, Geometric)

We discount the shape parameters of the Gamma/Beta distributions:

\[ \alpha_{\text{new}} = f \cdot \alpha_{\text{prior}} + \text{new data} \] \[ \beta_{\text{new}} = f \cdot \beta_{\text{prior}} + \text{new data} \]
Continuous Data (1D & 2D Normal)

We discount the Precision Matrix \( \Lambda = \Sigma^{-1} \). This naturally widens the confidence intervals before the new observation is incorporated:

\[ \Lambda_{\text{new}} = f \cdot \Lambda_{\text{prior}} + \Lambda_{\text{data}} \] \[ \boldsymbol{\mu}_{\text{new}} = \Lambda_{\text{new}}^{-1} (f \cdot \Lambda_{\text{prior}} \boldsymbol{\mu}_{\text{prior}} + \Lambda_{\text{data}} \mathbf{x}) \]
Multi-Category Data (Multinomial-Dirichlet)

We discount the entire \( \boldsymbol{\alpha} \) vector to allow for shifting user preferences:

\[ \alpha_{i, \text{new}} = f \cdot \alpha_{i, \text{prior}} + x_i \]

By sacrificing pure Bayesian optimality, we gain the agility to track moving targets, changing click-through rates, and dynamic server loads—the exact techniques used in modern Kalman Filters and Multi-Armed Bandits.

Real-World Applications

Each model in this series is tailored to a specific type of data. Here is how they are applied in the real world:

1. Poisson-Gamma: Discrete
Measures: Event counts in a fixed timeframe.
Use: Telecommunications (calls per minute), web analytics (server requests per second), and retail (customers entering a store per hour).
2. Binomial-Beta: Discrete
Measures: Success rates (proportions).
Use: The gold standard for A/B testing. Used to model click-through rates (CTR) on ads, email open rates, and medical treatment success probabilities.
3. Geometric-Beta: Discrete
Measures: Discrete time/steps (Failures before the 1st success).
Use: Modeling user patience. Examples include clicks before abandoning a webpage, products inspected before finding a defect, or number of throws missed before scoring.
4. Exponential-Gamma: Continuous
Measures: Continuous time (Time between events).
Use: Reliability engineering and system monitoring. Used to measure the time until a server crashes, the lifespan of a lightbulb, or the time between API requests.
5. Normal-Normal: Continuous
Measures: Continuous measurements & means.
Use: Industrial quality control (average temperature of a reactor), finance (average stock price), and healthcare (average heart rate of a patient).
6. Bivariate Normal: Continuous
Measures: 2D spatial coordinates (X, Y).
Use: Tracking moving objects in 2D space. Used in GPS navigation, radar systems tracking aircraft, and self-driving cars tracking surrounding vehicles.
7. Multinomial-Dirichlet: Discrete
Measures: Multi-category success rates.
Use: Used whenever there are more than two outcomes. Powers Natural Language Processing (topic modeling), email sorting (Spam, Promotions, Primary), and UI/UX tracking.
8. Negative Binomial: Discrete
Measures: Discrete time/steps (Failures before \(r\) successes).
Use: Used when a process must hit a specific target. Modeling accident-free months before a driver files 3 claims, or users seeing an ad before achieving 5 conversions. Negative Binomial is similar to Geometric Model, where the number of trial is not fixed, but instead of a single success, there is a specific number of successes that may occur in any order, hence the use of binomial coefficient. The negative binomial distribution gives you the probability of achieving a fixed number of successes (\(r\)) on a specific total number of trials (\(x\)) (or alternatively, having a specific number of failures before that \(r\)-th success). Wikipedia