AutoInt

Head Aggregation

Process of concatenating or averaging the outputs of all attention heads to form a unified representation before passing it to the next layer.

← Zurück