Efficient Transformers

Compressive Transformer

Extension of Transformer-XL that compresses old hidden memories into denser vectors to preserve long-term history. This compression enables efficient storage of extensive contextual information.

← Tillbaka