Variations of Recurrent Neural Network

There are various variations of RNN which depends on the type of input data provided and the type of output required, and also on the problem we are trying to solve using Recurrent Neural Networks. Each variation has a different RNN architecture and the implementation depends on this architecture.

Here is a representation of all the variations explained in an image below

Test Image

Many-to-many : Input units = Output Units

Example: Named Entity recognition

The number of input units $T_x$ is equal to the number of output units $T_y$

Many-to-many : Input units $T_x$ $\ne$ Output Units $T_y$

Example: Machine translation

Input: A sentence in french language Output: Translation in english language

Many-to-one

Example: Sentiment Classification

Input: A sentence with $T_x$ input units
Output: 0/1 (sigmoid) of [1,2,3,4,5]

One-to-many

Example: Music Generation

Input: Empty set or an integer (to represent Genre or Artist) Output: $y^{\langle 1 \rangle}$, $y^{\langle 2 \rangle}$, $y^{\langle 3 \rangle}$,….

One-to-one

Generic Neural Network

Written on February 14, 2018
]