Leaky ReLU Activation Function

Problem

Implement the Leaky ReLU activation function. Unlike standard ReLU which outputs 0 for negative inputs, Leaky ReLU allows a small, non-zero gradient (controlled by parameter alpha) for negative values.

Solution

import numpy as np

def leaky_relu(z, alpha=0.01):
    z = np.array(z, dtype=np.float64)
    return np.where(z > 0, z, alpha * z).tolist()

Explanation

1. Convert input to a NumPy array.
2. Use np.where to select: if the element is positive, keep it; otherwise multiply by alpha.
3. The small slope for negative values prevents "dying neurons" that can occur with standard ReLU.

Complexity

• Time: O(n) where n is the number of elements
• Space: O(n) for the output array