LoRA has become a key method for fine-tuning large models, but its parameter redundancy limits efficiency. This research introduces SeLoRA, which employs spectral encoding to reduce redundancy without sacrificing expressiveness, demonstrating improved performance and efficiency across various tasks like commonsense reasoning and code generation.
lora ✓
machine-learning ✓
parameter-redundancy ✓
spectral-encoding ✓
+ fine-tuning