← back

Bayesian Inference for Beta-Binomial Model

#213 · Statistics · Medium

⊣ Solve on deep-ml.com

Problem

Implement Bayesian inference for the Beta-Binomial model. Given a Beta prior with parameters alpha and beta, and observed binomial data (k successes out of n trials), compute the posterior distribution, posterior mean, and the 95% credible interval.

Solution

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
import math

def beta_binomial_posterior(k: int, n: int,
                            alpha_prior: float = 1.0,
                            beta_prior: float = 1.0) -> dict:
    # Posterior parameters (conjugate update)
    alpha_post = alpha_prior + k
    beta_post = beta_prior + (n - k)

    # Posterior mean
    mean = alpha_post / (alpha_post + beta_post)

    # Posterior mode (MAP estimate)
    if alpha_post > 1 and beta_post > 1:
        mode = (alpha_post - 1) / (alpha_post + beta_post - 2)
    else:
        mode = mean  # fallback

    # Posterior variance
    total = alpha_post + beta_post
    variance = (alpha_post * beta_post) / (total ** 2 * (total + 1))

    # 95% credible interval using beta quantile approximation
    lower = _beta_quantile(0.025, alpha_post, beta_post)
    upper = _beta_quantile(0.975, alpha_post, beta_post)

    return {
        "alpha_posterior": alpha_post,
        "beta_posterior": beta_post,
        "posterior_mean": round(mean, 6),
        "posterior_mode": round(mode, 6),
        "posterior_variance": round(variance, 6),
        "credible_interval_95": (round(lower, 4), round(upper, 4)),
    }

def _beta_quantile(p, a, b, tol=1e-8, max_iter=100):
    """Find x such that I_x(a, b) = p using bisection."""
    lo, hi = 0.0, 1.0
    for _ in range(max_iter):
        mid = (lo + hi) / 2
        if _beta_cdf(mid, a, b) < p:
            lo = mid
        else:
            hi = mid
        if hi - lo < tol:
            break
    return (lo + hi) / 2

def _beta_cdf(x, a, b):
    """Regularized incomplete beta function - simple numerical integration."""
    if x <= 0:
        return 0.0
    if x >= 1:
        return 1.0
    n_steps = 1000
    dx = x / n_steps
    total = 0.0
    ln_beta = _log_beta(a, b)
    for i in range(n_steps):
        t = (i + 0.5) * dx
        total += math.exp((a - 1) * math.log(t) + (b - 1) * math.log(1 - t) - ln_beta) * dx
    return total

def _log_beta(a, b):
    return math.lgamma(a) + math.lgamma(b) - math.lgamma(a + b)

Explanation

  1. Conjugate update: The Beta distribution is the conjugate prior for the Binomial likelihood. The posterior is Beta(alpha + k, beta + n - k).
  2. Posterior mean: alpha_post / (alpha_post + beta_post), the expected value of the Beta distribution.
  3. Posterior mode (MAP): (alpha_post - 1) / (alpha_post + beta_post - 2) when both parameters > 1.
  4. Credible interval: Use bisection on the Beta CDF (computed via numerical integration) to find the 2.5th and 97.5th percentiles.
  5. With a uniform prior (alpha=1, beta=1), the posterior mean is (k+1) / (n+2), a Laplace-smoothed estimate.

Complexity

  • Time: O(n_steps) for the numerical CDF integration per quantile query; O(1) for the posterior parameters
  • Space: O(1)