Implement Ridge Regression Loss Function

Problem

Implement the Ridge Regression loss function. Ridge regression adds an L2 penalty term to the ordinary least squares loss. Given predictions, true values, weights, and a regularization parameter lambda, compute the total loss.

Solution

import numpy as np

def ridge_loss(y_true, y_pred, weights, alpha):
    y_true = np.array(y_true, dtype=np.float64)
    y_pred = np.array(y_pred, dtype=np.float64)
    weights = np.array(weights, dtype=np.float64)

    n = len(y_true)
    mse = np.sum((y_true - y_pred) ** 2) / n
    penalty = alpha * np.sum(weights ** 2)

    return mse + penalty

Explanation

1. Compute the mean squared error (MSE) between true and predicted values.
2. Compute the L2 penalty as alpha * sum(weights^2).
3. The total Ridge loss is MSE plus the regularization penalty.
4. The penalty discourages large weight values, helping to prevent overfitting.

Complexity

• Time: O(n + p) where n is the number of samples and p is the number of weights
• Space: O(1) additional space beyond the inputs