Glosarium AI
Kamus lengkap Kecerdasan Buatan
Bidirectional Attention
Type of attention where each position can attend to all other positions in the sequence without restriction, allowing to capture complete contextual dependencies in both directions.
Residual Connections
Shortcut connections that add the input of a layer to its output, enabling deeper training by facilitating gradient flow and preserving information across multiple Transformer layers.
Attention Heads
Independent parallel subspaces in multi-head attention where each head learns different attention patterns, enabling the model to simultaneously capture various types of syntactic and semantic relationships.
Cross-Modal Attention
Extension of cross-attention where queries, keys, and values come from different modalities (text, image, audio), allowing multimodal models to align and fuse information between different sensory representations.
Attention Heatmap
Visualization of attention weights as a heatmap showing how the model distributes its attention across different parts of the input, an essential tool for the interpretability of Transformer models.