ExponentialLR Learning Rate Scheduler

Problem

Implement the ExponentialLR learning rate scheduler. The learning rate decays exponentially every epoch by multiplying with gamma.

Solution

def exponential_lr(initial_lr: float, gamma: float, epoch: int) -> float:
    return initial_lr * (gamma ** epoch)

Explanation

1. At each epoch, the learning rate is initial_lr * gamma^epoch.
2. Unlike StepLR which decays in discrete steps, ExponentialLR decays every single epoch.
3. For example, with initial_lr=0.1 and gamma=0.95, at epoch 10 the LR is 0.1 * 0.95^10 ≈ 0.0598.

Complexity

• Time: O(1)
• Space: O(1)