In machine learning, what does "overfitting" refer to?

Correct answer: When a model performs well on training data but poorly on new data

Option B is correct because overfitting occurs when a model learns the training data too precisely, including its noise and outliers, resulting in high accuracy on training examples but poor generalization to unseen data. Option A is incorrect because failure to converge describes an optimization problem during training, often related to learning rate or loss landscape issues, which is a distinct concept from overfitting. Option C is incorrect because inadequate feature engineering is a data preparation issue that typically leads to underfitting or high bias rather than overfitting. Option D is incorrect because having a very large training dataset generally reduces overfitting risk by exposing the model to more diverse examples, making it the opposite of what causes overfitting.