Linear Regression - Power Grid Optimization

Problem

Implement linear regression for power grid optimization. Given features (e.g., temperature, demand) and target values (power output), fit a linear regression model using the normal equation and return the predicted values and model coefficients.

Solution

import numpy as np

def linear_regression(X: np.ndarray, y: np.ndarray) -> tuple:
    # Add bias column
    m = X.shape[0]
    X_b = np.column_stack([np.ones(m), X])

    # Normal equation: theta = (X^T X)^(-1) X^T y
    theta = np.linalg.inv(X_b.T @ X_b) @ X_b.T @ y

    predictions = X_b @ theta
    return theta, predictions

Explanation

1. Bias term: Prepend a column of ones to the feature matrix so the model can learn an intercept.
2. Normal equation: Directly solve for the optimal weights using theta = (X^T X)^{-1} X^T y. This gives the closed-form least-squares solution.
3. Predictions: Multiply the augmented feature matrix by the learned weights to get predicted values.

The normal equation avoids iterative optimization (no learning rate or epochs needed) but requires the matrix X^T X to be invertible.

Complexity

• Time: O(n * d^2 + d^3) where n is the number of samples and d is the number of features
• Space: O(d^2) for the matrix computations