Implement Binary Cross-Entropy Loss

Problem

Implement Binary Cross-Entropy (BCE) Loss from scratch. Given true binary labels and predicted probabilities, compute the average loss.

Solution

BCE = -1/n * sum(y * log(p) + (1-y) * log(1-p)). Clip predictions to avoid log(0).

import math

def binary_cross_entropy(
    y_true: list[int],
    y_pred: list[float],
    epsilon: float = 1e-15,
) -> float:
    n = len(y_true)
    total_loss = 0.0

    for i in range(n):
        p = max(min(y_pred[i], 1.0 - epsilon), epsilon)
        loss = -(y_true[i] * math.log(p) + (1 - y_true[i]) * math.log(1 - p))
        total_loss += loss

    return round(total_loss / n, 6)

Explanation

1. For each sample, clip the predicted probability to [epsilon, 1 - epsilon] to prevent log(0).
2. Compute the per-sample loss: -[y * log(p) + (1-y) * log(1-p)].
3. When y=1, only the -log(p) term matters — penalizes low confidence in the positive class.
4. When y=0, only the -log(1-p) term matters — penalizes high confidence in the positive class.
5. Return the mean loss across all samples.

Complexity

• Time: O(n) where n is the number of samples
• Space: O(1) — constant extra storage