Model Fine-tuning: Difference between revisions
Jump to navigation
Jump to search
adding fine tune resources |
adding fine tune resources and time estimation suggestions |
||
| Line 1: | Line 1: | ||
===Fine-tune | ===Fine-tune Resources=== | ||
* [https://huggingface.co/docs/transformers/training Fine-tune a pre-trained model on HuggingFace] | * [https://huggingface.co/docs/transformers/training Fine-tune a pre-trained model on HuggingFace] | ||
* [https://platform.openai.com/docs/guides/fine-tuning Fine-tune documentation on OpenAI] | * [https://platform.openai.com/docs/guides/fine-tuning Fine-tune documentation on OpenAI] | ||
* | *[https://huggingface.co/blog/stackllama#stackllama-a-hands-on-guide-to-train-llama-with-rlhf StackLLaMA: A hands-on guide to train LLaMA with RLHF] | ||
*Parameter-Efficient Fine-Tuning (PEFT) -- An efficient fine-tuning approach to reduce computational resources and time cost | |||
**[https://huggingface.co/blog/peft Blog] | |||
**[https://github.com/huggingface/peft Github] | |||
=== Fine-tune time estimation (Based on previous Hackathon feedback) === | |||
* Fine-tuning ESM-2 model with 35M parameter takes ~4.5 hours | |||
* Fine-tuning ESM-2 model with 8M parameter takes ~2.5 hours | |||
Latest revision as of 01:41, 4 August 2023
Fine-tune Resources
[edit | edit source]- Fine-tune a pre-trained model on HuggingFace
- Fine-tune documentation on OpenAI
- StackLLaMA: A hands-on guide to train LLaMA with RLHF
- Parameter-Efficient Fine-Tuning (PEFT) -- An efficient fine-tuning approach to reduce computational resources and time cost
Fine-tune time estimation (Based on previous Hackathon feedback)
[edit | edit source]- Fine-tuning ESM-2 model with 35M parameter takes ~4.5 hours
- Fine-tuning ESM-2 model with 8M parameter takes ~2.5 hours