Deep Learning Glossary | Practice Test / Quiz / MCQs

Home > Deep Learning 101 > Quizzes > Deep Learning Glossary

Deep Learning Glossary

Fast practice, instant feedback. Timer auto-submits when time’s up.

Avg score: 0% • Most missed: “Alexnet”

Study First Challenge a Friend Stats

Deep Learning Glossary

Time left 00:00

25 Questions

1. Max-Pooling

A Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersRefers To The Technique Of Initializing A Network With Parameters From Another Task (Such As An Unsupervised Training Task) - And Then Updating These Parameters Based On The Task At Hand. For Example - Nlp Architecture Often Use Pre-Trained Word Embeddings Like Word2vec - And These Word Embeddings Are Then Updated During Training Based For A Specific Task Like Sentiment AnalysisThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification Tasks

2. Ilsvrc

3. Restricted Boltzmann Machine (Rbn)

Refers To The Technique Of Initializing A Network With Parameters From Another Task (Such As An Unsupervised Training Task) - And Then Updating These Parameters Based On The Task At Hand. For Example - Nlp Architecture Often Use Pre-Trained Word Embeddings Like Word2vec - And These Word Embeddings Are Then Updated During Training Based For A Specific Task Like Sentiment AnalysisThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient Descent

4. Fine-Tuning

A Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentRefers To The Technique Of Initializing A Network With Parameters From Another Task (Such As An Unsupervised Training Task) - And Then Updating These Parameters Based On The Task At Hand. For Example - Nlp Architecture Often Use Pre-Trained Word Embeddings Like Word2vec - And These Word Embeddings Are Then Updated During Training Based For A Specific Task Like Sentiment AnalysisA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification Tasks

5. Adadelta

A Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Gradient Descent Based Learning Algorithm That Adapts The Learning Rate Per Parameter Over Time. It Was Proposed As An Improvement Over Adagrad - Which Is More Sensitive To Hyperparameters And May Decrease The Learning Rate Too Aggressively. Adadelta It Is Similar To Rmsprop And Can Be Used Instead Of Vanilla SgdA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient Descent

6. Mnist

The Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersPerhaps Most Commonly Used Image Recognition Dataset. It Consists Of 60 - 000 Training And 10 - 000 Test Examples Of Handwritten Digits. Each Image Is 28ã—28 Pixels Large. State Of The Art Models Typically Achieve Accuracies Of 99.5% Or Higher On The Test Set

7. Affine Layer

A Fully-Connected Layer In A Neural Network. Affine Means That Each Neuron In The Previous Layer Is Connected To Each Neuron In The Current Layer. In Many Ways - This Is The "Standard" Layer Of A Neural Network. Affine Layers Are Often Added On Top Of The Outputs Of Convolutional Neural Networks Or Recurrent Neural Networks Before Making A Final Prediction. An Affine Layer Is Typically Of The Form Y = F(Wx + B) Where X Are The Layer Inputs - W The Parameters - B A Bias Vector - And F A Nonlinear Activation FunctionA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient Descent

8. Softmax

The Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentTypically Used To Convert A Vector Of Raw Scores Into Class Probabilities At The Output Layer Of A Neural Network Used For Classification. It Normalizes The Scores By Exponentiating And Dividing By A Normalization Constant. If We Are Dealing With A Large Number Of Classes - A Large Vocabulary In Machine Translation For Example - The Normalization Constant Is Expensive To Compute. There Exist Various Alternatives To Make The Computation More Efficient - Including Hierarchical Softmax Or Using A Sampling-Based Loss Such As Nce

9. Noise-Contrastive Estimation (Nce)

A Sampling Loss Typically Used To Train Classifiers With A Large Output Vocabulary. Calculating The Softmax Over A Large Number Of Possible Classes Is Prohibitively Expensive. Using Nce - We Can Reduce The Problem To Binary Classification Problem By Training The Classifier To Discriminate Between Samples From The "Real" Distribution And An Artificially Generated Noise DistributionThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional Layers

10. Recursive Neural Network

A Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Generalization Of Recurrent Neural Networks To A Tree-Like Structure. The Same Weights Are Applied At Each Recursion. Just Like Rnns - Recursive Neural Networks Can Be Trained End-To-End Using Backpropagation. While It Is Possible To Learn The Tree Structure As Part Of The Optimization Problem - Recursive Neural Networks Are Often Applied To Problem That Already Have A Predefined Structure - Like A Parse Tree In Natural Language ProcessingA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient Descent

11. Lstm

The Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentLong Short-Term Memory Networks Were Invented To Prevent The Vanishing Gradient Problem In Recurrent Neural Networks By Using A Memory Gating Mechanism. Using Lstm Units To Calculate The Hidden State In An Rnn We Help To The Network To Efficiently Propagate Gradients And Learn Long-Range Dependencies

12. Gru

A Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Simplified Version Of An Lstm Unit With Fewer Parameters. Just Like An Lstm Cell - It Uses A Gating Mechanism To Allow Rnns To Efficiently Learn Long-Range Dependency By Preventing The Vanishing Gradient Problem. The Gru Consists Of A Reset And Update Gate That Determine Which Part Of The Old Memory To Keep Vs. Update With New Values At The Current Time Step

13. Adam

An Adaptive Learning Rate Algorithm Similar To Rmsprop - But Updates Are Directly Estimated Using A Running Average Of The First And Second Moment Of The Gradient And Also Include A Bias Correction TermThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient Descent

14. Deep Dream

The Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersA Technique Invented By Google That Tries To Distill The Knowledge Captured By A Deep Convolutional Neural Network. The Technique Can Generate New Images - Or Transform Existing Images And Give Them A Dreamlike Flavor - Especially When Applied Recursively

15. Seq2seq

The Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersReads A Sequence (Such As A Sentence) As An Input And Produces Another Sequence As An Output. It Differs From A Standard Rnn In That The Input Sequence Is Completely Read Before The Network Starts Producing Any Output. Typically - Seq2seq Models Are Implemented Using Two Rnns - Functioning As Encoders And Decoders. Neural Machine Translation Is A Typical Example Of A Seq2seq ModelA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient Descent

16. Multilayer Perceptron (Mlp(

A Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Feedforward Neural Network With Multiple Fully-Connected Layers That Use Nonlinear Activation Functions To Deal With Data Which Is Not Linearly Separable. An Mlp Is The Most Basic Form Of A Multilayer Neural Network - Or A Deep Neural Networks If It Has More Than 2 LayersThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional Layers

17. Vanishing Gradient Problem

The Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentArises In Very Deep Neural Networks - Typically Recurrent Neural Networks - That Use Activation Functions Whose Gradients Tend To Be Small (In The Range Of 0 From 1). Because These Small Gradients Are Multiplied During Backpropagation - They Tend To "Vanish" Throughout The Layers - Preventing The Network From Learning Long-Range Dependencies. Common Ways To Counter This Problem Is To Use Activation Functions Like Relus That Do Not Suffer From Small Gradients - Or Use Architectures Like Lstms That Explicitly Combat Vanishing Gradients. The Opposite Of This Problem Is Called The Exploding Gradient Problem

18. Resnet

The Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentWon The Ilsvrc 2015 Challenge. These Networks Work By Introducing Shortcut Connection Across Stacks Of Layers - Allowing The Optimizer To Learn "Easier" Residual Mappings Instead Of The More Complicated Original Mappings. These Shortcut Connections Are Similar To Highway Layers - But They Are Data-Independent And Don't Introduce Additional Parameters Or Training Complexity. Resnets Achieved A 3.57% Error Rate On The Imagenet Test SetA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional Layers

19. Vgg

A Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersRefers To Convolutional Neural Network Model That Secured The First And Second Place In The 2014 Imagenet Localization And Classification Tracks - Respectively. The Vgg Model Consist Of 16-19 Weight Layers And Uses Small Convolutional Filters Of Size 3ã—3 And 1ã—1

20. Bidirectional Rnn

A Type Of Neural Network That Contains Two Rnns Going Into Different Directions. The Forward Rnn Reads The Input Sequence From Start To End - While The Backward Rnn Reads It From End To Start. The Two Rnns Are Stacked On Top Of Each Others And Their States Are Typically Combined By Appending The Two Vectors. Bidirectional Rnns Are Often Used In Natural Language Problems - Where We Want To Take The Context From Both Before And After A Word Into Account Before Making A PredictionA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification Tasks

21. Pooling

The Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentSee Max-Pooling Or Average-Pooling

22. Highway Layer

A Type Of Neural Network Layer That Uses A Gating Mechanism To Control The Information Flow Through A Layer. Stacking Multiple Highway Layers Allows For Training Of Very Deep Networks. Highway Layers Work By Learning A Gating Function That Chooses Which Parts Of The Inputs To Pass Through And Which Parts To Pass Through A Transformation Function - Such As A Standard Affine Layer For Example. The Basic Formulation Of A Highway Layer Is T H(X) + (1 - T) X - Where T Is The Learned Gating Function With Values Between 0 And 1 - H(X) Is An Arbitrary Input Transformation And X Is The Input. Note That All Of These Must Have The Same SizeThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional Layers

23. Gradient Clipping

A Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Technique To Prevent Exploding Gradients In Very Deep Networks - Typically Recurrent Neural Networks. There Exist Various Ways To Perform Gradient Clipping - But The A Common One Is To Normalize The Gradients Of A Parameter Vector When Its L2 Norm Exceeds A Certain Threshold According To New_Gradients = Gradients * Threshold / L2_Norm(Gradients)A Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional LayersThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification Tasks

24. Autoencoder

The Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Neural Network Model Whose Goal Is To Predict The Input Itself - Typically Through A "Bottleneck" Somewhere In The Network. By Introducing A Bottleneck - We Force The Network To Learn A Lower-Dimensional Representation Of The Input - Effectively Compressing The Input Into A Good Representation. Autoencoders Are Related To Pca And Other Dimensionality Reduction Techniques - But Can Learn More Complex Mappings Due To Their Nonlinear Nature. A Wide Range Of Autoencoder Architectures Exist - Including Denoising Autoencoders - Variational Autoencoders - Or Sequence AutoencodersA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional Layers

25. Average-Pooling

A Pooling Technique Used In Convolutional Neural Networks For Image Recognition. It Works By Sliding A Window Over Patches Of Features - Such As Pixels - And Taking The Average Of All Values Within The Window. It Compresses The Input Representation Into A Lower-Dimensional RepresentationThe Imagenet Large Scale Visual Recognition Challenge Evaluates Algorithms For Object Detection And Image Classification At Large Scale. It Is The Most Popular Academic Challenge In Computer Vision. Over The Past Years - Deep Learning Techniques Have Led To A Significant Reduction In Error Rates - From 30% To Less Than 5% - Beating Human Performance On Several Classification TasksA Type Of Probabilistic Graphical Model That Can Be Interpreted As A Stochastic Artificial Neural Network. Rbns Learn A Representation Of The Data In An Unsupervised Manner. An Rbn Consists Of Visible And Hidden Layer - And Connections Between Binary Neurons In Each Of These Layers. Rbns Can Be Efficiently Trained Using Contrastive Divergence - An Approximation Of Gradient DescentA Pooling Operations Typically Used In Convolutional Neural Networks. A Max-Pooling Layer Selects The Maximum Value From A Patch Of Features. Just Like A Convolutional Layer - Pooling Layers Are Parameterized By A Window (Patch) Size And Stride Size. For Example - We May Slide A Window Of Size 2ã—2 Over A 10ã—10 Feature Matrix Using Stride Size 2 - Selecting The Max Across All 4 Values Within Each Window - Resulting In A New 5ã—5 Feature Matrix. Pooling Layers Help To Reduce The Dimensionality Of A Representation By Keeping Only The Most Salient Information - And In The Case Of Image Inputs - They Provide Basic Invariance To Translation (The Same Maximum Values Will Be Selected Even If The Image Is Shifted By A Few Pixels). Pooling Layers Are Typically Inserted Between Successive Convolutional Layers

⚡ Recently practiced quizzes in this topic

Live quiz activity

Deep Learning through Transformative Pedagogy Deep Learning Glossary 2 Deep Learning Knowledge Test Artificial Neural Networks

25 Questions

❤ If you liked Fatskills, consider supporting us by checking out The Life Manuals You Never Got.

About | Explore | User Guide | Topics | Subjects | Doubt Solver | Career Aptitude Test | Answers | Free Tools | OSHA Basics Quiz | What Should We Know?
Privacy | Terms |

Without work one finishes nothing. - Ralph Waldo Emerson
© 2026 Fatskills.com

All trademarks, logos and brand names are the property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, trademarks and brands does not imply endorsement.

25 Questions

❤ If you liked Fatskills, consider supporting us by checking out The Life Manuals You Never Got.

About | Explore | User Guide | Topics | Subjects | Doubt Solver | Career Aptitude Test | Answers | Free Tools | OSHA Basics Quiz | What Should We Know? Privacy | Terms |

Without work one finishes nothing. - Ralph Waldo Emerson© 2026 Fatskills.com

All trademarks, logos and brand names are the property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, trademarks and brands does not imply endorsement.

About | Explore | User Guide | Topics | Subjects | Doubt Solver | Career Aptitude Test | Answers | Free Tools | OSHA Basics Quiz | What Should We Know?
Privacy | Terms |

Without work one finishes nothing. - Ralph Waldo Emerson
© 2026 Fatskills.com