Słownik AI
Kompletny słownik sztucznej inteligencji
Attention Matrix
Square matrix representing attention weights between all pairs of elements in a sequence, visualizing the interdependence patterns learned by the model.
Encoder-Decoder Attention
Mechanism where the decoder attends to encoder outputs, enabling generation of sequences conditioned on a source sequence in seq2seq models.
Masked Self-Attention
Self-attention with masking of future positions to prevent elements from seeing subsequent information, essential in text generation tasks.
Softmax Normalization
Activation function transforming attention scores into probability distribution, ensuring that the sum of attention weights equals 1 for each position.
Attention Head
Sub-component of multi-head attention performing an independent attention calculation with its own parameters, capturing a specific type of sequential relationship.
Attention Dropout
Regularization applied to attention weights during training, randomly deactivating some connections to prevent overfitting of attention patterns.
Multi-Scale Attention
Attention variant simultaneously processing dependencies at different temporal or spatial scales, combining varied receptive fields for hierarchical understanding.