Explain the difference between cross-validation and bootstrapping.

Official answer

Cross-validation repeatedly trains on part of a dataset and evaluates on held-out observations. Its usual purpose is predictive model assessment or selection. In k-fold validation, each fold is held out once; preprocessing and tuning must respect that separation.

The ordinary nonparametric bootstrap repeatedly draws a sample of the original size with replacement and recomputes a statistic. Its distribution across resamples can estimate uncertainty, such as the variability of a sample mean. Some observations appear several times and others are absent from a resample.

Neither procedure is assumption-free. The ordinary row bootstrap relies on an appropriate independent sampling model and can perform poorly for some statistics or small samples. Clustered or time-dependent data may need cluster or block resampling. Cross-validation likewise needs splits that match dependence and deployment.

Use cross-validation to compare predictive procedures; use a suitable bootstrap when estimating sampling uncertainty is the objective. There are bootstrap methods for prediction error too, but scoring a model on the same bootstrap observations used to fit it is not an honest held-out evaluation.

Sources

scikit-learn: cross-validation; SciPy: bootstrap confidence intervals.