Machine Learning 101 Practice Test: Ensemble Learning — Flashcards

Quiz questions on ensemble learning, covering error-correcting output codes, model combination schemes, boosting weak learnability, the AdaBoost algorithm, and stacking.

Ensemble learning is a machine learning technique that combines the predictions of multiple models to improve performance and reduce the risk of choosing a poor model. The goal is to achieve better performance with the ensemble of models than with any individual model. Ensemble learning works best when the base models are not correlated. For example, you can train different models such as linear models, decision trees, and neural nets on different datasets or features. 

Common ensemble learning methods include:

Bagging: Also known as bootstrap aggregation, this method involves several weak learners learning the data pattern independently and simultaneously. The output of each weak learner is averaged to generate the final output of the model.
Boosting: This method involves training homogenous weak learners sequentially such that a base model depends on the previously fitted base models. All these base learners are then combined in a very adaptive way to obtain an ensemble model. Boosting assigns equal weights to all data points, but reduces the weight of correctly classified samples, and increases the weight of incorrectly classified samples. This process continues until the model as a whole gives strong predictions.
Stacking: Also known as stacked generalization, this method uses the combined output of multiple models to build a new model with improved performance. In stacking, an algorithm takes the outputs of sub-models as input and attempts to learn how to best combine the input predictions to make a better output prediction.