Implement the Huber Loss Function

Problem

Implement the Huber Loss function (also called Smooth L1 Loss). It behaves like MSE for small errors and like MAE for large errors, controlled by a threshold parameter delta.

Solution

import numpy as np

def huber_loss(y_true: np.ndarray, y_pred: np.ndarray,
               delta: float = 1.0) -> float:
    y_true = np.array(y_true, dtype=float)
    y_pred = np.array(y_pred, dtype=float)
    residual = np.abs(y_true - y_pred)
    loss = np.where(
        residual <= delta,
        0.5 * residual ** 2,
        delta * residual - 0.5 * delta ** 2
    )
    return float(np.mean(loss))

Explanation

1. Compute the absolute residual |y_true - y_pred| for each element.
2. For residuals <= delta (small errors), use quadratic loss: 0.5 * residual^2 (like MSE).
3. For residuals > delta (large errors), use linear loss: delta * |residual| - 0.5 * delta^2 (like MAE, shifted to be continuous).
4. Return the mean loss across all elements.
5. The Huber loss is differentiable everywhere and less sensitive to outliers than MSE.

Complexity

• Time: O(n) where n is the number of elements
• Space: O(n) for the residual and loss arrays