Fine-tuning is the process of taking pre-trained models and further training them on smaller, domain-specific datasets. This process transforms general-purpose models into specialized ones, bridging the gap between generic pre-trained models and the unique requirements of particular applications.
Why use Fine-Tuning over General LLMs?
Reasons why fine-tuning models for tasks in new domains are crucial
• Domain-Specific Adaptation: Fine-tuning allows adaptation to the nuances and characteristics of a new domain, enhancing performance in domain-specific tasks.
• Shifts in Data Distribution: Fine-tuning helps align the model with the distribution of new data, addressing shifts in data characteristics and improving performance on specific tasks.
• Continual Learning: Fine-tuning supports continual learning by allowing models to adapt to evolving data and user requirements over time. It enables models to stay relevant and effective in dynamic environments
Let’s understand this with a case study example:
The case study referred here uses a general model (Llama 2 in this case) for fine tuning on Covid 19 Patient Data for accurate diagnosis and treatment recommendations. They curated supervised training datasets (more on this later) to help the general LLM understand health patterns, suggest treatments and respond effectively to clinical queries.
The domain-specific adaptation in this case showed how LLMs like LLaMA can be transformed into powerful, context-aware tools for specialized sectors such as healthcare. The case highlights the broader potential of fine-tuning general-purpose LLMs to meet domain-specific challenges.
Types of Fine-Tuning
Fine-tuning a language can be done using two main approaches: unsupervised and supervised. Let’s understand each of them briefly with some examples
• Unsupervised Fine-Tuning Method: In unsupervised learning method the data is passed unlabeled to extract patterns and structures without explicit labels. This method is relevant when there is a need to update the knowledge base of an LLM without modifying its behavior
Example:
A pioneering Large Language Model for Law
This research paper introduces SaulLM-7B model tailored for the legal domain. It leverages Mistral 7B as its base model and it is trained on an extensive English legal corpus of unlabeled legal documents, including case laws, statutes, contracts and legal opinions.
Methodology:
o The base model used in this case is Mistral 7B model
o This research uses a vast collection of unlabeled legal documents for training data, including case law, statutes, contracts, and legal opinions.
o The model adapts the specific language patterns, terminology, and structures prevalent in legal texts by continuous pre-training on the legal corpus.
End Results:
This approach enhanced SaulLM-7B’s efficiency after fine-tuning in understanding and generating texts for legal domain without relying on labeled data. It outperformed previous models in tasks such as legal document summarization and legal question answering, showcasing the effectiveness of unsupervised domain adaptation in specialized fields.
• Supervised Fine-Tuning Method: It involves having labeled data where model is trained on examples with corresponding desired outputs. During supervised fine-tuning, the pre-trained LLM is fine-tuned on this labeled dataset using supervised learning techniques. The supervised fine-tuning process allows the model to learn task-specific patterns and nuances present in the labeled data. By adapting its parameters according to the specific data distribution and task requirements, the model becomes specialized in performing well on the target task.
Example:
Supervised Fine-Tuning Model for Enhanced Financial Data Analysis
Researchers of this paper developed a specialized LLM model named Raven to improve tabular data analysis in the financial sector. They used supervised learning techniques for fine-tuning Meta’s Llama-2 13B chat model. They tailored Raven to handle complex financial tasks that require precise reasoning and data interpretation.
Methodology:
o The base model used in this case is Llama-2 13B chat model
o They used labeled datasets for training data which comprised of financial question-answer pairs that enable model to learn domain-specific knowledge and reasoning patterns
o The tailored model, Raven, effectively acted as both “task router” and “task retriever” with the capability to utilize external tools for tasks like calculations and data retrieval
End Results:
This research shows that Raven is fine-tuned using supervised learning method demonstrated a 35.2% improvement over the base model. This showcases the effectiveness of supervised fine-tuning in addressing the nuanced requirements of financial data analysis.
References
1) https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week3_finetuning_llms.md
2) https://medium.com/mantisnlp/supervised-fine-tuning-customizing-llms-a2c1edbf22c3
3) https://www.spaceo.ai/case-study/fine-tuning-llama-2/
4) https://arxiv.org/html/2403.03883v1
5) https://arxiv.org/abs/2401.15328