AI Glossary
The complete dictionary of Artificial Intelligence
Hard Attention to the Task (HAT)
Technique using learned binary attention masks to select subnetworks dedicated to each task, ensuring complete parameter isolation between different sequential tasks.
SupSup
Continual learning method superimposing weight masks on top of a shared neural network, allowing the addition of new tasks without modifying previously learned weights.
Network Gating
Mechanism controlling the activation of different parts of the neural network based on the current task, using learnable gates to isolate task-specific parameters.
Task-specific Subnetworks
Specialized subnetworks identified or created within a main network, each optimized for a particular task while remaining functionally independent from other subnetworks.
Binary Mask Learning
Process of learning binary masks that determine which network parameters are active for a given task, ensuring strict separation between parameter sets of different tasks.
Progressive Network Expansion
Strategy where the network progressively expands with new units or layers dedicated to each new task, while keeping existing parameters unchanged to preserve prior knowledge.
Neural Network Pruning for Continual Learning
Selective application of neural pruning to free up parameters in an existing network, thereby creating space for learning new tasks without disrupting performance of previous tasks.
Isolated Parameter Allocation
Systematic process of assigning neural parameters to specific tasks in a way that ensures no shared parameters can be modified when learning new tasks.
Subnetwork Discovery
Automatic technique identifying optimal subnetworks within a larger architecture, each specialized for a particular task while maintaining the functional integrity of the overall system.
Mask-based Continual Learning
Continual learning paradigm using parameter masks to segment the network's weight space into regions dedicated to different tasks, thus eliminating the need for parameter replication.
Fixed Backbone Networks
Architecture where the fundamental layers of the network remain frozen after initial learning, with only classification heads or specialized subnetworks added for new tasks.
Adaptive Parameter Selection
Intelligent mechanism dynamically selecting parameters to use or modify when learning a new task, based on their relevance and usage history.
Continual Learning with Parameter Isolation
Continual learning theoretical framework based on the principle of strict parameter isolation between tasks, contrasting with regularization-based or rehearsal-based approaches.
Task-aware Parameter Routing
Intelligent routing system directing information through specific parameters based on task identity, ensuring functional isolation while optimizing resource utilization.