top

Five Important Techniques That You Should Know About Deep Learning

Deep Learning is a process of data mining which uses architectures of a deep neural network, which are specific types of artificial intelligence and machine learning algorithms that have become extremely important in the past few years. It's anticipated that may deep learning applications will influence your life soon. Deep learning allows us to teach machines how to complete complex tasks without explicitly programming them to do so. We're entering the era of artificial intelligence and machine learning. A future where machines will be doing many of the jobs that humans are currently doing today. In the past, we'd have to explicitly program a computer step by step how to solve a problem. This involved a lot of if-then statements, for loops, and logical operations. In the future, machines will teach themselves how to solve problems, we just need to provide the data. Let us learn about the techniques that allow deep learning to solve a variety of problems.1. Fully Connected Neural NetworksFully Connected Feedforward Neural Networks are the standard network architecture used in most basic neural network applicationsFully connected means that each neuron in the preceding layer is connected to every neuron in the subsequent layer. And feedforward means that neurons in any preceding layer are only ever connected to the neurons in a subsequent layer. Each neuron in a neural network contains an activation function that changes the output of a neuron given its input. These activation functions are:Linear function - It is a straight line that essentially multiplies the input by a constant value.Non-Linear Function Sigmoid function- It is an S-shaped curve ranging from 0 to 1.Hyperbolic tangent (tanH) function- It is an S-shaped curve ranging from -1 to +1Rectified linear unit(ReLU) function- It is a piecewise function that outputs a 0 if the input is less than a certain value, or linear multiple if the input is greater than a certain value.Each type of activation function has pros and cons, so we use them in various layers in a deep neural network based on the problem each is designed to solve. Non-linearity is what allows deep neural networks to model complex functions.We can create networks with various inputs, various outputs, various hidden layers, various neurons per hidden layer, and a variety of activation functions. These numerous combinations allow us to create a variety of powerful deep neural networks that can solve a wide array of problems. The more neurons we add to each hidden layer, the wider the network becomes. In addition, the more hidden layers we add, the deeper the network becomes. However, each neuron we add increases the complexity and thus the processing power necessary to train a neural network.2. Convolutional Neural NetworksConvolutional Neural Networks(CNN) is a type of deep neural network architecture designed for specific tasks like image classification. CNNs were inspired by the organization of neurons in the visual cortex of the animal brain. As a result, they provide some very interesting features that are useful for processing certain types of data like images, audio, and video.A CNN is composed of an input layer. However, for basic image processing, this input is typically a two- dimensional array of neurons which correspond to the pixels of an image. It also contains an output layer which is typically a one- dimensional set of output neurons.CNN uses a combination of sparsely connected convolution layers, which perform image processing on their inputs. In addition, they contain down sampling layers called pooling layers to further reduce the number of neurons necessary in subsequent layers of the network. And finally, CNNs typically contain one or more fully connected layers to connect our pooling layer to our output layer.Convolution is a technique that allows us to extract visual features from an image in small chunks. Each neuron in a convolution layer is responsible for a small cluster of neurons in the preceding layer.It contains filters or kernel that determines the cluster of neurons.Filters mathematically modify the input of a convolution to help it detect certain types of features in the image. They can return the unmodified image, blur the image, sharpen the image, detect edges etc. This is done by multiplying the original image values by a convolution matrix.Pooling, also known as subsampling or downsampling reduces the number of neurons in the previous convolution layer while still retaining the most important information. There are different types of pooling that can be performed. For example, taking the average of each input neuron, the sum, or the maximum value.We can also reverse this architecture to create what is known as a deconvolution neural network. These networks perform the inverse of a convolutional network i.e. Rather than taking an image and converting it into a prediction value, these networks take an input value and attempt to produce an image instead.CNNs work well for a variety of tasks including image recognition, image processing, image segmentation, video analysis, and natural language processing.3. Recurrent Neural NetworkThe recurrent neural network (RNN), unlike feedforward neural networks, can operate effectively on sequences of data with variable input length.This means that RNNs uses knowledge of its previous state as an input for its current prediction, and we can repeat this process for an arbitrary number of steps allowing the network to propagate information via its hidden state through time. This is essentially like giving a neural network a short-term memory. This feature makes RNNs very effective for working with sequences of data that occur over time, For example, the time-series data, like changes in stock prices, a sequence of characters, like a stream of characters being typed into a mobile phone.Let's imagine we're creating a recurrent neural network to predict the next letter a person is likely to type based on the previous letters they've already typed. The letter that a user just typed as well as all of the previous letters is important in predicting the next letter. First, the user types the letter h, so our network might predict that the next letter is i based on the previous training to predict hi. Then, the user types the letter e, so our network uses both the new letter e plus the state of the first hidden neuron in order to compute our next prediction y because of the high frequency of occurrences of the word hey in our training dataset. Adding the letter l might predict the word help, and adding another l would predict the letter o, which would match the word our user intended to type, which is hello.The two variants on the basic RNN architecture that help solve a common problem with training RNNs are Gated RNNs, and Long Short-Term Memory RNNs (LSTMs). Both of these variants use a form of memory to help make predictions in sequences over time. The main difference between a Gated RNN and an LSTM is that the Gated RNN has two gates to control its memory: an Update gate and a Reset gate, while an LSTM has three gates: an Input gate, an Output gate, and a Forget gate.RNNs work well for applications that involve a sequence of data that changes over time. These applications include natural language processing, speech recognition, language translation, image captioning, conversation modeling, and visual Q&A. 4. Generative Adversarial NetworksThe Generative Adversarial Network(GAN) is a combination of two deep learning neural networks: a Generator Network, and a Discriminator Network. The Generator Network produces synthetic data, and the Discriminator Network tries to detect if the data that it's seeing is real or synthetic.These two networks are adversaries in the sense that they're both competing to beat one another. The Generator is trying to produce synthetic data indistinguishable from real data, and the Discriminator is trying to become progressively better at detecting fake data. For example, imagine we want to create a neural network that generates synthetic images. First, we'd acquire a library of real-world images that we can use to provide real images for the image detector network. Next, we'd create an Image Generator network to produce synthetic images. This would typically be a deconvolution neural network.Then we'd create an Image Detector network to detect real images versus fake images. This would typically be a convolutional neural network. At first, the generator would essentially create random noise as it learns how to create images that can fool the detector. In addition, the detector would only have roughly 50/50 accuracy when predicting real versus fake images. However, with each training iteration, the generator gets progressively better at generating real images, and the detector gets progressively better at detecting fake images. If you let these networks compete with one another for long enough, the generator begins producing fake images that approximate real images. Generative Adversarial Networks have gained quite a bit of popularity in recent years. Some of their applications include image generation, image enhancement, text generation, speech synthesis, new drug discovery etc. 5. Deep Reinforcement LearningReinforcement learning involves an agent interacting with an environment. The agent is trying to achieve a goal of some kind within the environment. The environment has a state, which the agent can observe. The agent has actions that it can take, which modify the state of the environment, and the agent receives reward signals when it achieves a goal of some kind. The objective of the agent is to learn how to interact with its environment in such a way that allows it to achieve its goals.Deep reinforcement learning is the application of reinforcement learning to train deep neural networks. Like our previous deep neural networks, we have an input layer, an output layer, and multiple hidden layers. However, our input is the state of the environment. For example, a car trying to get its passengers to their destination the inputs are the position, speed, and direction; our output is a series of possible actions like speed up, slow down, turn left, or turn right. In addition, we're feeding our rewards signal into the network so that we can learn to associate what actions produce positive results given a specific state of the environment. This deep neural network attempts to predict the expected future reward for each action, given the current state of the environment. It then chooses whichever action's predicted to have the highest potential future reward, and performs that action.Some examples of Deep Reinforcement Learning Applications are games, including board games like chess and card games like poker. Autonomous vehicles, like self-driving cars, and autonomous drones. Robotics, like teaching robots how to walk, and how to perform manual tasks. Management tasks, including inventory management, resource allocation, and logistics; and financial tasks, including investment decisions, portfolio design, and asset pricing.Hope, this deep learning tutorial helped you to learn some valuable information and got an idea about various Deep Learning techniques. Each technique is useful in its own way and is put to practical use in various applications daily. I hope you will be able to put this information to great use.
Rated 4.0/5 based on 39 customer reviews
Normal Mode Dark Mode

Five Important Techniques That You Should Know About Deep Learning

Rohit Goyal
Blog
09th May, 2018
Five Important Techniques That You Should Know About Deep Learning

Deep Learning is a process of data mining which uses architectures of a deep neural network, which are specific types of artificial intelligence and machine learning algorithms that have become extremely important in the past few years. It's anticipated that may deep learning applications will influence your life soon. Deep learning allows us to teach machines how to complete complex tasks without explicitly programming them to do so. We're entering the era of artificial intelligence and machine learning. A future where machines will be doing many of the jobs that humans are currently doing today. In the past, we'd have to explicitly program a computer step by step how to solve a problem. This involved a lot of if-then statements, for loops, and logical operations. In the future, machines will teach themselves how to solve problems, we just need to provide the data. Let us learn about the techniques that allow deep learning to solve a variety of problems.

1. Fully Connected Neural Networks

Fully Connected Feedforward Neural Networks are the standard network architecture used in most basic neural network applications

 Fully Connected Neural Networks
Fully connected means that each neuron in the preceding layer is connected to every neuron in the subsequent layer. And feedforward means that neurons in any preceding layer are only ever connected to the neurons in a subsequent layer. Each neuron in a neural network contains an activation function that changes the output of a neuron given its input. These activation functions are:

  • Linear function - It is a straight line that essentially multiplies the input by a constant value.
  • Non-Linear Function
  •  Sigmoid function- It is an S-shaped curve ranging from 0 to 1.
  • Hyperbolic tangent (tanH) function- It is an S-shaped curve ranging from -1 to +1
  • Rectified linear unit(ReLU) function- It is a piecewise function that outputs a 0 if the input is less than a certain value, or linear multiple if the input is greater than a certain value.

Each type of activation function has pros and cons, so we use them in various layers in a deep neural network based on the problem each is designed to solve. Non-linearity is what allows deep neural networks to model complex functions.
activation functionWe can create networks with various inputs, various outputs, various hidden layers, various neurons per hidden layer, and a variety of activation functions. These numerous combinations allow us to create a variety of powerful deep neural networks that can solve a wide array of problems. The more neurons we add to each hidden layer, the wider the network becomes. In addition, the more hidden layers we add, the deeper the network becomes. However, each neuron we add increases the complexity and thus the processing power necessary to train a neural network.


2. Convolutional Neural Networks

Convolutional Neural Networks(CNN) is a type of deep neural network architecture designed for specific tasks like image classification. CNNs were inspired by the organization of neurons in the visual cortex of the animal brain. As a result, they provide some very interesting features that are useful for processing certain types of data like images, audio, and video.
Convolutional Neural Networks
A CNN is composed of an input layer. However, for basic image processing, this input is typically a two- dimensional array of neurons which correspond to the pixels of an image. It also contains an output layer which is typically a one- dimensional set of output neurons.CNN uses a combination of sparsely connected convolution layers, which perform image processing on their inputs. In addition, they contain down sampling layers called pooling layers to further reduce the number of neurons necessary in subsequent layers of the network. And finally, CNNs typically contain one or more fully connected layers to connect our pooling layer to our output layer.CNN layersConvolution is a technique that allows us to extract visual features from an image in small chunks. Each neuron in a convolution layer is responsible for a small cluster of neurons in the preceding layer.

It contains filters or kernel that determines the cluster of neurons.

Filters mathematically modify the input of a convolution to help it detect certain types of features in the image. They can return the unmodified image, blur the image, sharpen the image, detect edges etc. This is done by multiplying the original image values by a convolution matrix.

Pooling, also known as subsampling or downsampling reduces the number of neurons in the previous convolution layer while still retaining the most important information. There are different types of pooling that can be performed. For example, taking the average of each input neuron, the sum, or the maximum value.

We can also reverse this architecture to create what is known as a deconvolution neural network. These networks perform the inverse of a convolutional network i.e. Rather than taking an image and converting it into a prediction value, these networks take an input value and attempt to produce an image instead.

CNNs work well for a variety of tasks including image recognition, image processing, image segmentation, video analysis, and natural language processing.

3. Recurrent Neural Network


Recurrent Neural Network
The recurrent neural network (RNN), unlike feedforward neural networks, can operate effectively on sequences of data with variable input length.

This means that RNNs uses knowledge of its previous state as an input for its current prediction, and we can repeat this process for an arbitrary number of steps allowing the network to propagate information via its hidden state through time. This is essentially like giving a neural network a short-term memory. This feature makes RNNs very effective for working with sequences of data that occur over time, For example, the time-series data, like changes in stock prices, a sequence of characters, like a stream of characters being typed into a mobile phone.

Let's imagine we're creating a recurrent neural network to predict the next letter a person is likely to type based on the previous letters they've already typed. The letter that a user just typed as well as all of the previous letters is important in predicting the next letter. First, the user types the letter h, so our network might predict that the next letter is i based on the previous training to predict hi. Then, the user types the letter e, so our network uses both the new letter e plus the state of the first hidden neuron in order to compute our next prediction y because of the high frequency of occurrences of the word hey in our training dataset. Adding the letter l might predict the word help, and adding another l would predict the letter o, which would match the word our user intended to type, which is hello.

The two variants on the basic RNN architecture that help solve a common problem with training RNNs are Gated RNNs, and Long Short-Term Memory RNNs (LSTMs). Both of these variants use a form of memory to help make predictions in sequences over time. The main difference between a Gated RNN and an LSTM is that the Gated RNN has two gates to control its memory: an Update gate and a Reset gate, while an LSTM has three gates: an Input gate, an Output gate, and a Forget gate.

RNNs work well for applications that involve a sequence of data that changes over time. These applications include natural language processing, speech recognition, language translation, image captioning, conversation modeling, and visual Q&A.


 4. Generative Adversarial Networks

The Generative Adversarial Network(GAN) is a combination of two deep learning neural networks: a Generator Network, and a Discriminator Network. The Generator Network produces synthetic data, and the Discriminator Network tries to detect if the data that it's seeing is real or synthetic.

These two networks are adversaries in the sense that they're both competing to beat one another. The Generator is trying to produce synthetic data indistinguishable from real data, and the Discriminator is trying to become progressively better at detecting fake data. For example, imagine we want to create a neural network that generates synthetic images. First, we'd acquire a library of real-world images that we can use to provide real images for the image detector network. Next, we'd create an Image Generator network to produce synthetic images. This would typically be a deconvolution neural network.
Generative adversarial netwrorkThen we'd create an Image Detector network to detect real images versus fake images. This would typically be a convolutional neural network. At first, the generator would essentially create random noise as it learns how to create images that can fool the detector. In addition, the detector would only have roughly 50/50 accuracy when predicting real versus fake images. However, with each training iteration, the generator gets progressively better at generating real images, and the detector gets progressively better at detecting fake images. If you let these networks compete with one another for long enough, the generator begins producing fake images that approximate real images. Generative Adversarial Networks have gained quite a bit of popularity in recent years. Some of their applications include image generation, image enhancement, text generation, speech synthesis, new drug discovery etc. 

5. Deep Reinforcement Learning

Reinforcement learning involves an agent interacting with an environment. The agent is trying to achieve a goal of some kind within the environment. The environment has a state, which the agent can observe. The agent has actions that it can take, which modify the state of the environment, and the agent receives reward signals when it achieves a goal of some kind. The objective of the agent is to learn how to interact with its environment in such a way that allows it to achieve its goals.

Deep reinforcement learning is the application of reinforcement learning to train deep neural networks. Like our previous deep neural networks, we have an input layer, an output layer, and multiple hidden layers. However, our input is the state of the environment. For example, a car trying to get its passengers to their destination the inputs are the position, speed, and direction; our output is a series of possible actions like speed up, slow down, turn left, or turn right. In addition, we're feeding our rewards signal into the network so that we can learn to associate what actions produce positive results given a specific state of the environment. This deep neural network attempts to predict the expected future reward for each action, given the current state of the environment. It then chooses whichever action's predicted to have the highest potential future reward, and performs that action.
Reinforcement learningSome examples of Deep Reinforcement Learning Applications are games, including board games like chess and card games like poker. Autonomous vehicles, like self-driving cars, and autonomous drones. Robotics, like teaching robots how to walk, and how to perform manual tasks. Management tasks, including inventory management, resource allocation, and logistics; and financial tasks, including investment decisions, portfolio design, and asset pricing.

Hope, this deep learning tutorial helped you to learn some valuable information and got an idea about various Deep Learning techniques. Each technique is useful in its own way and is put to practical use in various applications daily. I hope you will be able to put this information to great use.

Rohit

Rohit Goyal

Blog Author

Rohit is a Full Stack Developer and Programmer by profession, passionate about technologies and is constantly looking for better ways to write code. When not making loud noises on his mechanical keyboard, he can be found reading books or playing music.

Leave a Reply

Your email address will not be published. Required fields are marked *

SUBSCRIBE OUR BLOG

Follow Us On

Share on

other Blogs

20% Discount