Although bidirectional RNNs are kind of a state of the art technologies, there are several limitations of using them. Since bidirectional RNNs operate in both positive and negative directions, they are extremely slow since the gradients have a long dependency chain.

Also, bidirectional RNNs are used in very specific applications, such as filling in the missing word, machine translation, and so on. Another major issue with this algorithm is that it is challenging to train because of memory-bandwidth-bound computations.