Parameter Regularization

Orthogonal Gradient Descent (OGD)

Method that projects the gradient of the new task onto the space orthogonal to the gradient subspaces of previous tasks. This projection guarantees that learning new tasks does not interfere with directions important for past performance.

← Geri