LoRA (Low-Rank Adaptation)

LoRA rank hyperparameter

Parameter determining the dimension of the low-rank matrices in LoRA decomposition, controlling the trade-off between expressiveness and efficiency. Typical ranks range from 4 to 64 depending on the complexity of the adaptation task.

← Geri