Train, validation and test sets
Training fits model parameters; validation guides choices; a held-out test estimates performance on unseen cases.
Loading video…
Training fits model parameters; validation guides choices; a held-out test estimates performance on unseen cases. Split by patient or other independent unit before fitting imputation, scaling or feature selection. Cross-validation repeats train/validation splits within development data; the final test should remain untouched. Evaluating on training cases measures memorization as well as learning.
Worked example
If two EEG windows come from the same person, put both in the same partition. Fit a scaler on training patients, then apply those learned parameters to validation and test patients.