Theory of Kernel Generalization

Experiments and figures of the paper "Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks" https://arxiv.org/abs/2006.13198

Paper experiments:

Synthetic Data - Gaussian Regression:

Synthetic Data - NTK Regression and NN:

Synthetic Data - Synthetic Kernel Regression:

Real Data - Gaussian Regression:

Numerical Computation of NTK Eigenvalues:

Recreate the figures using precomputed data:

Fig. 1 and Fig. SI.10:

Fig. 2 and Fig. SI.1:

Fig. 3, Fig. 4, Fig. SI.2:

Fig. 5 and Fig. SI.5:

Fig. 6, Fig. SI.8, Fig. SI.9:

Fig. SI.6 and Fig. SI.7: and