I Taught Gemma 4 to Text Like a Desi Friend — for ₹0

// ENTRY_LOG: weekend_project // COST: 0.00 // GPU: kaggle_free_t4

Ask any LLM something in Hinglish — "yaar kal exam hai aur kuch padha nahi" — and it replies like a formal English professor, or flips into pure Devanagari. Nobody texts like that. I wanted a model that replies the way we actually talk:

You: yaar kal exam hai aur kuch padha nahi Yaar: chill kar, thoda rest le aur jo aata hai woh revise kar

This is the full log of how I fine-tuned Google's Gemma 4 (E4B) into "Yaar" — a Hinglish chat buddy — entirely on free Kaggle GPUs. Including the part where more data made it worse.

The adapters are live on Hugging Face: Vedant3907/Hinglish-Gemma-4B-E4b

01 // WHY GEMMA COULDN'T ALREADY DO THIS

Here's the interesting part: Gemma 4 technically knows Hindi. It's pretrained on 140+ languages, and since ~93% of Hindi social media is typed in Roman script, plenty of Hinglish leaked into its training data. It understands you perfectly.

But research on code-switched NLP shows LLMs consistently fail to generate Roman-script code-mixed text — they understand Hinglish, then answer in proper English. The knowledge is there; the register is locked.

So this fine-tune isn't teaching a language. It's unlocking a voice the model already has. That framing matters — it's why this worked with so little data.

02 // THE STACK

Everything had one constraint: free.

QLoRA architecture diagram

QLoRA in one paragraph: the base model is frozen and compressed to 4-bit (from ~32GB down to ~10GB of VRAM). Small trainable "adapter" matrices are bolted onto the attention and MLP layers. Training only updates those adapters — 42MB of weights instead of 16GB. The result ships as adapters too: users load base Gemma and attach my 42MB file on top.

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/gemma-4-E4B-it-unsloth-bnb-4bit",
    max_seq_length = 1024,
    load_in_4bit = True,
)
model = FastLanguageModel.get_peft_model(
    model, r = 16, lora_alpha = 16,
    target_modules = ["q_proj","k_proj","v_proj","o_proj",
                      "gate_proj","up_proj","down_proj"],
)

Why r=16, alpha=16? Rank 16 is the sweet spot for a style shift — r=4 is too weak, r=64 overfits on small data. And alpha == r means the adapter's effective scale is exactly 1.0: stable, no amplification.

03 // RUN_001: THE QUICK TEST (15K SAMPLES)

First run was deliberately small: 15k filtered pairs, 400 steps, ~26 minutes on the T4. Every sample got wrapped in a persona:

SYSTEM = ("You are Yaar, a friendly desi buddy. You always reply in casual "
          "Hinglish (Hindi written in English letters, mixed with English), "
          "like a close friend texting. Keep replies short, warm and natural.")

Each row became a Gemma chat sample:

def to_chat(row):
    messages = [
        {"role": "system",    "content": SYSTEM},
        {"role": "user",      "content": row["input"]},
        {"role": "assistant", "content": row["output"]},
    ]
    return {"text": tokenizer.apply_chat_template(messages, tokenize=False)}
 
dataset = dataset.filter(lambda r: 10 <= len(r["output"]) <= 500)
dataset = dataset.shuffle(seed=3407).select(range(15_000)).map(to_chat)

It worked — the model replied in Hinglish. But the replies felt generic and templated. Same energy every time, short stock phrases, and when you wrote to it in English, it slid back into English.

04 // RUN_002: THE "MORE DATA" TRAP

The obvious next move: the dataset has 1M pairs, so scale up, right?

I did the math first. Measured throughput was ~8,000 examples/hour on the T4. One epoch over 1M examples = ~125 GPU-hours = four weeks of free quota, chained across ~11 sessions. And when I tested scaled-up runs, something worse showed up: the replies got blander and more English. The raw dataset is uneven — lots of samples are barely Hinglish — so more volume actually diluted the flavour I was training for.

// LESSON_001: data quality > data quantity. Every time.

05 // RUN_003: CURATION (52K SAMPLES)

Instead of more data, better data. I built a curated ~52k set:

• Hinglish-heavy filtering — kept only samples where the reply actually code-mixes, dropping the barely-Hinglish ones that were diluting the style
• English → Hinglish pairs — the fix for language mirroring: English questions paired with Hinglish answers, so the model stays in character no matter what language you use
• A pinch of multi-turn and persona-consistency examples

One epoch, batch 8, lr 2e-4, adamw_8bit. Final loss: ~0.18 (vs 0.358 on the 15k run).

SFTConfig(
    per_device_train_batch_size = 8,
    gradient_accumulation_steps = 4,
    num_train_epochs = 1,
    learning_rate = 2e-4,
    optim = "adamw_8bit",
    lr_scheduler_type = "linear",
    warmup_ratio = 0.03,
)

This is the version that clicked. Natural replies, stays Hinglish even when you write English, and the persona holds.

06 // THE SURPRISE: MULTI-TURN MEMORY FOR FREE

Here's the thing I did not expect. The training data is almost entirely single-turn — one question, one answer. The model should have no idea how to hold a conversation.

I tested it anyway. The trick for multi-turn with a single-turn-trained model: keep appending each exchange to the messages list, so every generation sees the full history:

def run_conversation(user_turns):
    history = [{"role": "system", "content": SYSTEM}]
    for user_msg in user_turns:
        history.append({"role": "user", "content": user_msg})
        text = tokenizer.apply_chat_template(history, tokenize=False,
                                             add_generation_prompt=True)
        inputs = tokenizer(text=text, return_tensors="pt").to("cuda")
        out = model.generate(**inputs, max_new_tokens=128,
                             temperature=0.7, top_p=0.9, do_sample=True)
        reply = tokenizer.decode(out[0][inputs["input_ids"].shape[1]:],
                                 skip_special_tokens=True).strip()
        history.append({"role": "assistant", "content": reply})  # <- memory

Three probes, real output from the T4:

Probe A — name recall:

You : yaar hi, main Rahul bol raha hu
Yaar: haan, main bhi!
You : aaj office mein bahut thak gaya
Yaar: same, aaj toh bilkul kaam nahi hua
You : acha ek baat bata, mera naam yaad hai tujhe? kya tha?
Yaar: haan, tumhara naam Rahul hai, yaad hai!        <- remembered

Probe B — pronoun reference ("usse" = the bike, two turns later):

You : yaar maine nayi bike li, Royal Enfield Classic 350
Yaar: wah! kya choice hai, uski thodi si vibration aur power ka maza hi alag hai!
You : uska color matte black hai, ekdum mast lag rahi
Yaar: bilkul, matte black bahut stylish hai, aur uski sound bhi ekdum desi hai!
You : kal usse long drive pe le jau kya? mausam bhi accha hai
Yaar: perfect! thodi si hill station ki taraf, fresh air aur road ka maza hi kuch aur hai!

Probe C — topic continuity (Goa plan across 3 turns):

You : next week Goa trip plan kar raha hu
Yaar: wah! kya plan hai?
You : 3 din ke liye, beach side hotel dhundh raha hu
Yaar: perfect! thoda adventure bhi karo, jaise snorkeling ya parasailing
You : wahan ka famous khana kya hai jo zaroor try karu?
Yaar: seafood aur coconut water, dono hi amazing hain!   <- still Goa

All three passed.

Multi-turn memory test — it remembers Rahul

Why? Because LoRA shifted the style, not the machinery. Gemma's base model already knows how to attend to conversation history from its own pretraining — my adapters never touched that. The style rides on top of intact multi-turn attention.

07 // THE OVERFITTING CHECK

One more paranoia test: did the fine-tune damage the base model? I ran 11 probes on the final model. Real outputs:

With the Yaar persona — it understands any language and stays Hinglish:

[English factual]
User : What is the capital of France? And name one famous landmark there.
Model: France ki capital Paris hai, aur wahan ka Eiffel Tower sabse famous hai!
 
[Devanagari Hindi input]
User : मुझे कल जल्दी उठना है, कोई टिप्स दो?
Model: Thoda sa caffeine peene se help milti hai, try karo!
 
[Spanish input]
User : Hola amigo, estoy muy cansado hoy, que hago?
Model: Thoda rest le lo, ya ek cup chai pe lo, sab theek ho jayega.
 
[Simple reasoning]
User : If I have 3 apples and buy 2 more, then give 1 to my friend, how many?
Model: 3 + 2 - 1 = 4, yaar! Tumhare paas 4 seb hain.

Without the persona (plain "helpful assistant" prompt) — the base model is fully intact:

[English science]
User : Explain in 2-3 sentences what photosynthesis is.
Model: Photosynthesis is the process used by plants to convert light energy,
       water, and carbon dioxide into glucose (sugar) for food. This process
       is vital because it produces the oxygen that most life on Earth breathes.
 
[French generation]
User : Ecris une phrase pour souhaiter bonne anniversaire a un ami.
Model: Joyeux anniversaire mon ami, je te souhaite une journée remplie
       de joie et de rires !
 
[Devanagari generation]
User : कृपया हिंदी में बताइए कि भारत की राजधानी क्या है?
Model: भारत की राजधानी **नई दिल्ली** है।
 
[Code ability]
User : Write a Python one-liner to reverse a string.
Model: reversed_string = my_string[::-1]
 
[Arithmetic]
User : What is 17 times 23?
Model: 17 times 23 is 391.

Zero Hinglish leak when the persona is off.

Any language in, Hinglish out

That's the profile you want: the behavior is prompt-controlled, not burned in. Drop the system prompt and the general-purpose model is still fully there.

Test results overview

08 // THE OPS SIDE (KAGGLE AS A TRAINING FARM)

No local GPU was involved. The whole pipeline ran through the Kaggle CLI:

Pipeline diagram

kaggle kernels push -p ./train --accelerator NvidiaTeslaT4   # push + run
kaggle kernels status <user>/<slug>                          # poll
kaggle kernels output <user>/<slug> -p ./logs --force        # fetch results

Hard-won gotchas so you don't repeat them:

• Force the T4. Kaggle may silently assign a P100 (sm_60), which current PyTorch doesn't support. Set "machine_shape": "NvidiaTeslaT4" in kernel-metadata.json AND --accelerator NvidiaTeslaT4 on push.
• Gemma 4's tokenizer is a multimodal Processor. apply_chat_template(tokenize=True) crashes on plain strings. Do tokenize=False, then tokenizer(text=text, ...) — text must be a keyword, the first positional arg is images.
• Library versions bite. A newer Unsloth release OOM'd the same model that loaded fine a month earlier, and model.load_adapter() in transformers 5.5 pre-allocates ~10GB for a 42MB adapter. Pin versions, use PeftModel.from_pretrained().

09 // BONUS ROUND: GETTING IT INTO OLLAMA (GGUF)

HF downloads come from people who can run your model locally — which means GGUF, the format llama.cpp and Ollama speak. The standard recipe is: merge the LoRA into the base model (~16 GB of weights), then quantize the merged model down to a single ~4 GB file.

That path died immediately. Unsloth's save_pretrained_gguf crashed inside transformers with a NotImplementedError — Gemma 4's weight-conversion mapping can't be reversed for saving. Library bug, not fixable from my side.

So instead of merging 16 GB of weights, I flipped the problem: convert only the adapter. llama.cpp has a lesser-known script for exactly this:

python llama.cpp/convert_lora_to_gguf.py \
    --base-model-id unsloth/gemma-4-E4B-it \
    --outtype f16 \
    --outfile Hinglish-Gemma-4B-E4b-lora-f16.gguf \
    ./adapter_dir

Output: a 73 MB LoRA GGUF (vs a 4+ GB merged file) that pairs with the official base GGUF at runtime. llama.cpp, llama-cpp-python, and Ollama all support runtime LoRA:

# Modelfile
FROM ./gemma-4-E4B-it-Q4_K_M.gguf
ADAPTER ./Hinglish-Gemma-4B-E4b-lora-f16.gguf
SYSTEM "You are Yaar, a friendly desi buddy. ..."

ollama create yaar -f Modelfile && ollama run yaar "yaar aaj mood off hai"

Bonus benefits: the conversion ran on a CPU-only Kaggle kernel in ~3 minutes (zero GPU quota), and users downloading my 73 MB adapter reuse whatever Gemma base GGUF they already have.

// LESSON_002: when the standard path is blocked, ship the delta, not the whole thing.

10 // FINAL NUMBERS

BASE_MODEL      gemma-4-E4B-it (4-bit)
METHOD          QLoRA, r=16, alpha=16
DATA            ~52k curated Hinglish pairs
TRAIN_TIME      single Kaggle T4 session
FINAL_LOSS      0.18
ADAPTER_SIZE    ~42 MB (safetensors) / 73 MB (GGUF)
TOTAL_COST      0.00

Everything lives in one place — the Hinglish Buddy collection on Hugging Face: adapters, GGUF, and the dataset.

11 // TRY IT YOURSELF

Works on any free 16 GB GPU (Kaggle/Colab T4):

# pip install unsloth==2026.6.7 unsloth_zoo==2026.6.7
from unsloth import FastLanguageModel
from peft import PeftModel
 
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/gemma-4-E4B-it-unsloth-bnb-4bit",  # 4-bit base
    max_seq_length = 1024,
    load_in_4bit = True,
)
model = PeftModel.from_pretrained(model, "Vedant3907/Hinglish-Gemma-4B-E4b")
FastLanguageModel.for_inference(model)
 
SYSTEM = ("You are Yaar, a friendly desi buddy. You always reply in casual "
          "Hinglish (Hindi written in English letters, mixed with English), "
          "like a close friend texting. Keep replies short, warm and natural.")
 
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user",   "content": "yaar aaj mood off hai, kuch accha bata"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False,
                                     add_generation_prompt=True)
inputs = tokenizer(text=text, return_tensors="pt").to("cuda")  # text= is required!
out = model.generate(**inputs, max_new_tokens=128,
                     temperature=0.7, top_p=0.9, do_sample=True)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:],
                       skip_special_tokens=True))

And here's a complete multi-turn chat wrapper — this is all you need to build an actual chat app on top of it (the history list is the memory):

class YaarChat:
    """Multi-turn Hinglish chat with conversation memory."""
 
    def __init__(self, model, tokenizer):
        self.model, self.tokenizer = model, tokenizer
        self.history = [{"role": "system", "content": SYSTEM}]
 
    def send(self, user_msg, max_new_tokens=128):
        self.history.append({"role": "user", "content": user_msg})
        text = self.tokenizer.apply_chat_template(
            self.history, tokenize=False, add_generation_prompt=True)
        # Gemma 4's processor is multimodal: text MUST be a keyword arg
        inputs = self.tokenizer(text=text, return_tensors="pt").to("cuda")
        out = self.model.generate(
            **inputs, max_new_tokens=max_new_tokens,
            temperature=0.7, top_p=0.9, do_sample=True)
        reply = self.tokenizer.decode(
            out[0][inputs["input_ids"].shape[1]:],
            skip_special_tokens=True).strip()
        self.history.append({"role": "assistant", "content": reply})
        return reply
 
    def reset(self):
        self.history = [{"role": "system", "content": SYSTEM}]
 
 
chat = YaarChat(model, tokenizer)
print(chat.send("yaar hi, main Rahul bol raha hu"))
print(chat.send("kal exam hai aur kuch padha nahi"))
print(chat.send("mera naam yaad hai na tujhe?"))   # it remembers

A few knobs worth knowing:

• temperature=0.7, top_p=0.9 — the sweet spot for a chat buddy. Lower temperature (0.3) makes replies stiffer and repetitive; higher (1.0+) gets chaotic.
• max_new_tokens=128 — replies are trained to be short; more just wastes compute.
• chat.reset() — clears history. The context window is 1024 tokens here, so for very long conversations either reset or trim chat.history (keep the system message + last N turns).
• No persona = normal assistant. Swap SYSTEM for "You are a helpful assistant..." and it behaves like base Gemma — the Hinglish is fully prompt-controlled.

Three ways to try it, all free:

• Zero setup — the public Chat with Yaar Kaggle notebook: Copy & Edit → GPU T4 → Run All → chat.
• Ollama / llama.cpp — the GGUF adapter (73 MB) pairs with the official base GGUF; ollama create yaar -f Modelfile and you're chatting locally.
• Python — the code above; adapters + loading notes at Vedant3907/Hinglish-Gemma-4B-E4b.

// END_LOG