Machine Consciousness
Machine Learning
A parameter-efficient fine-tuning (PEFT) technique freezing base model weights and training injected low-rank matrices.
Detailed Explanation
LoRA (Low-Rank Adaptation) is a fine-tuning method that adjusts pre-trained models by injecting trainable low-rank matrices into existing layers while keeping the original weights frozen. This approach reduces the number of trainable parameters, enabling efficient adaptation with less computational resources and memory, making it ideal for customizing large models for specific tasks without extensive retraining.
Use Cases
- •Fine-tuning a large language model for domain-specific sentiment analysis efficiently with minimal computational resources.
Related Terms
Other terms in the Machine Learning category