Fine-Tuning Mixtral-8x7B: A Deep Dive into a Multi-Node Slurm and FSDP Workflow

Training or fine-tuning large language models like Mixtral-8x7B is a significant challenge due to their large size. Sometimes, to make these …

Fine-tuning Gemma 3 on an A4 Slurm Cluster

Note: This post has been adapted into a formal tutorial on the Google Cloud documentation site.

Overview

This post demonstrates how to fine-tune the Gemma 3 language model on a multi-node, …

Fine-tuning Gemma 3 on an A3 Mega Slurm Cluster

Note: This post has been adapted into a formal tutorial on the Google Cloud documentation site.

Overview

This post demonstrates how to fine-tune the Gemma 3 language model on a multi-node, …