Machine Learning 101 Practice Test: Stochastic Gradient Descent — Flashcards

Quiz on optimization algorithms, specifically focusing on Stochastic Gradient Descent (SGD), its variants, the standard Gradient Descent Algorithm, and Subgradient Descent.

Stochastic Gradient Descent (SGD) is a gradient-based optimization algorithm that finds the optimal parameter configuration for a machine learning algorithm. It's a variant of the Gradient Descent algorithm. 

SGD iteratively updates a model's parameters one sample or batch at a time. This makes SGD models computationally efficient and able to handle large datasets. 
SGD is one of three types of gradient descent learning algorithms, along with batch gradient descent and mini-batch gradient descent. 

Here are some more details about SGD:

Gradient descent
A numerical algorithm that finds the lowest values of a function. It's a general-purpose algorithm that can be used to minimize differentiable functions.

Batch gradient descent
An iterative algorithm that updates the model parameters after processing the entire training dataset. It guarantees convergence to the global minimum, but can be computationally expensive and slow for large datasets.