← back

Maximum Likelihood Estimation for Gaussian Distribution

#337 · Statistics · Medium

⊣ Solve on deep-ml.com

Problem

Implement Maximum Likelihood Estimation (MLE) for the parameters of a Gaussian (normal) distribution. Given observed data, estimate the mean and variance that maximize the likelihood of observing the data.

Solution

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
from typing import List, Dict

def gaussian_mle(data: List[float]) -> Dict[str, float]:
    n = len(data)
    if n == 0:
        raise ValueError("Data cannot be empty")

    # MLE for mean: sample mean
    mean = sum(data) / n

    # MLE for variance: average squared deviation (not Bessel-corrected)
    variance = sum((x - mean) ** 2 for x in data) / n

    std_dev = variance ** 0.5

    return {
        "mean": round(mean, 4),
        "variance": round(variance, 4),
        "std_dev": round(std_dev, 4)
    }

def gaussian_log_likelihood(
    data: List[float],
    mean: float,
    variance: float
) -> float:
    import math
    if variance <= 0:
        raise ValueError("Variance must be positive")

    n = len(data)
    ll = -n / 2 * math.log(2 * math.pi * variance)
    ll -= sum((x - mean) ** 2 for x in data) / (2 * variance)
    return round(ll, 4)

Explanation

  1. The MLE for the mean of a Gaussian is the sample mean: mu_hat = (1/n) * sum(x_i).
  2. The MLE for the variance is the average squared deviation from the mean: sigma^2_hat = (1/n) * sum((x_i - mu_hat)^2). Note this uses n, not n-1 (MLE is biased for variance).
  3. The log-likelihood function is: -n/2 log(2pisigma^2) - sum((x_i - mu)^2) / (2sigma^2). These MLE estimates maximize this function.

Complexity

  • Time: O(n) where n is the number of data points
  • Space: O(1) extra space
← #336#338 →