Question

What does it mean to say that Gaussian kernel regression approaches nearest-neighbor regression as h +0? Why does it do so? Is this true for all kinds of kernel regression?

Answer 1

Solution: Regression function å las = { di W (mi, x) where wa, se) is nearest - neighbor regression defined as Wise si n nearest neighbor of se defined as Whi, k) = { o í otherwise The kernel regression function is Food = { di K«:, x) { K &;X! Here Krik) depends only on the distance xi-n. Rinal=h be any positive number known as bandwidth. As a fixed bandwidth h, kernel segression is generally not consistent. If huo as n mo then we get consistency we tend to get spikier function with the Gaussian Kernel at least it tends towards the nearest regression. It is due to bandwidth usually matters a lot more than the kernel. It puts us back to toughly where we were with k-NN regression, heeding to control the degree of smoothing without knowing how smooth ru really is. It is not true for all kinds of kernel regression.