Interactive Polynomial Regression Regularization

This interactive demo illustrates how regularization helps to control model complexity in polynomial regression. Adjust the sliders to see how the polynomial degree, regularization strength (λ), and data noise affect the model's fit.

Polynomial Degree:
15

Regularization Strength (λ):
1.0000e-7

Noise Level:
0.5

Number of Samples:
15

Regularization Type: Ridge (L2) Lasso (L1) Unregularized

Model Fit Visualization

Coefficient Magnitudes

Training and Validation Error

Model Performance

Coefficient	Value	Term

Understanding Regularization

Mathematical Formulation

In polynomial regression, we fit a model of the form:

$$f(x) = w_0 + w_1x + w_2x^2 + \ldots + w_nx^n$$

Loss Functions:

Unregularized: $$\mathcal{L} = \sum_{i=1}^{m} (y_i - f(x_i))^2 $$

Ridge Regularization (L2): $$\mathcal{L} = \sum_{i=1}^{m} (y_i - f(x_i))^2 + \lambda \sum_{j=1}^{n} w_j^2 $$

Lasso Regularization (L1): $$\mathcal{L} = \sum_{i=1}^{m} (y_i - f(x_i))^2 + \lambda \sum_{j=1}^{n} |w_j| $$

The regularization parameter λ controls the strength of the penalty. Higher values enforce stronger regularization, resulting in smaller coefficients.

Key Concepts

Why Regularization Matters:

Overfitting: High-degree polynomials can perfectly fit training data but generalize poorly
Variance-Bias Tradeoff: Regularization increases bias but reduces variance
Numerical Stability: Prevents exploding coefficients and improves conditioning

Comparing Regularization Types:

Ridge (L2): Shrinks all coefficients toward zero, but rarely to exactly zero
Lasso (L1): Promotes sparsity by forcing some coefficients to exactly zero (feature selection)
No Regularization: Fits the data as closely as possible, potentially overfitting

Try it: Increase the polynomial degree and observe how unregularized models start to overfit. Then increase λ to see how regularization smooths the curve and stabilizes predictions.