Binary Classification with Logistic Regression

Problem

Implement binary classification using logistic regression. Given features and labels, compute predictions using the sigmoid function applied to a linear combination of features, and return predicted probabilities and class labels.

Solution

import numpy as np

def logistic_regression_predict(X: np.ndarray, weights: np.ndarray, bias: float) -> tuple:
    z = X @ weights + bias
    probabilities = 1 / (1 + np.exp(-np.clip(z, -500, 500)))
    predictions = (probabilities >= 0.5).astype(int)
    return probabilities, predictions

def logistic_regression(X: np.ndarray, y: np.ndarray, lr: float = 0.01, epochs: int = 1000) -> tuple:
    n_samples, n_features = X.shape
    weights = np.zeros(n_features)
    bias = 0.0

    for _ in range(epochs):
        z = X @ weights + bias
        pred = 1 / (1 + np.exp(-np.clip(z, -500, 500)))

        dw = (X.T @ (pred - y)) / n_samples
        db = np.mean(pred - y)

        weights -= lr * dw
        bias -= lr * db

    return weights, bias

Explanation

1. Sigmoid function: Maps any real number to (0, 1) via 1 / (1 + exp(-z)), interpreting the output as a probability.
2. Linear model: Compute z = Xw + b then apply sigmoid to get predicted probabilities.
3. Training: Minimize binary cross-entropy loss using gradient descent. The gradient for weights is X^T (predictions - y) / n and for bias is mean(predictions - y).
4. Classification: Threshold probabilities at 0.5 to get binary class predictions.

Complexity

• Time: O(epochs n d) where n is samples and d is features
• Space: O(d) for model parameters