Can We Use Gradient Norm As a Measure of Generalization Error for Model
measure Given a set of Ntraining samplesfx 1 x 2 x 3 x Ng a deep learning model and a loss function L x based on the sample x the generalization gap Gis defined as G = E x˘X L x 1 N XN i=1 L x i 1 where Xis defined as the distribution of data and E x˘XL x refers to the expected loss To