Bidirectional RNNs
Tutorial
Bidirectional RNNs
Process sequences forwards and backwards simultaneously.
Bidirectional RNNs
Standard RNNs read text from left-to-right. A Bidirectional RNN (BiRNN) reads text simultaneously forward and backward, allowing the network to understand the context of a word using both the words that came before it and after it.
Why Bi-Directional Context Matters
Consider the sentence:
"He said, 'Teddy bears are on sale!'" vs "He said, 'Teddy Roosevelt was a president.'"
A forward-only RNN seeing "Teddy" doesn't know if it's a toy or a person. A BiRNN looks ahead to see "bears" or "Roosevelt", completely disambiguating the context instantly.
Level 1 — Bi-LSTM in Keras
Wrapping an LSTM in Bidirectional
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Bidirectional, LSTM, Dense, Embedding
model = Sequential([
Embedding(input_dim=5000, output_dim=64),
# The Bidirectional wrapper duplicates the layer: one forward, one backward.
# The outputs are concatenated, so 64 units x 2 = 128 dimension output.
Bidirectional(LSTM(64)),
Dense(1, activation='sigmoid')
])