ozertuu/bge-m3-food-projection-head

🍽️ BGE-M3 Food Projection Head

A lightweight 1024-dimensional Neural Domain Adapter (8.4 MB) trained on top of frozen BAAI/bge-m3 representations.

It projects general multilingual text embeddings into a specialized culinary semantic space, aligning Turkish and international dish names, cooking styles, and ingredients while pushing apart raw items, canned goods, and confusing modifiers.

Developed for the EatBetter multimodal meal recognition and nutrition engine.

---

💡 Key Highlights

• Base Model: BAAI/bge-m3 (560M parameters) — Kept 100% Frozen.
• Adapter Size: 2.1 Million parameters (8.4 MB / model.safetensors).
• Why Frozen Base? Prevents catastrophic forgetting of multilingual grammar while adding domain-specific contrastive alignment for gastronomy.
• Trained on: 30,001 culinary triplets from ozertuu/turkish-food-contrastive-triplets.

---

🏗️ How It Works

Input Text ("Soslu Kuru Fasulye")
             │
             ▼
┌──────────────────────────────────────────────┐
│  BAAI/bge-m3 Base Encoder (Frozen, 560M)     │
└──────────────────────┬───────────────────────┘
                       │ 1024-d Base Embedding (x)
                       ▼
┌──────────────────────────────────────────────┐
│  FoodProjectionHead (8.4 MB Residual Adapter)│
│  • Linear(1024, 1024) ──► LayerNorm ──► GELU │
│  • Linear(1024, 1024)                        │
│  • Output = Normalize(x + 0.5 * MLP(x))      │
└──────────────────────┬───────────────────────┘
                       │
                       ▼
   1024-d Culinary-Aligned Food Vector (z)

---

🎯 Semantic Separation Examples

Food Query A	Food Query B	Relationship	Similarity Score
Kuru Fasulye	Soslu etli kuru fasulye yemeği	Same Dish (Gastronomic Match)	0.94 (High)
Kuru Fasulye	Kuru fasulye konservesi / tohumu	Dish vs Raw / Canned Ingredient	0.41 (Separated)
Kuşbaşılı Pide	Kuşbaşı Et	Composite Bakery vs Pure Meat	0.38 (Separated)
Menemen	Domatesli Biberli Yumurta	Traditional Recipe Equivalence	0.91 (High)

---

🚀 Quickstart & Usage

import torch
from sentence_transformers import SentenceTransformer
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

# 1. Define Lightweight Adapter
class FoodProjectionHead(torch.nn.Module):
    def __init__(self, dim=1024):
        super().__init__()
        self.net = torch.nn.Sequential(
            torch.nn.Linear(dim, dim),
            torch.nn.LayerNorm(dim),
            torch.nn.GELU(),
            torch.nn.Linear(dim, dim)
        )
    def forward(self, x):
        return torch.nn.functional.normalize(x + 0.5 * self.net(x), p=2, dim=-1)

# 2. Load Base Model & Download Adapter
device = "cuda" if torch.cuda.is_available() else ("mps" if torch.backends.mps.is_available() else "cpu")
base_model = SentenceTransformer("BAAI/bge-m3", device=device)

adapter = FoodProjectionHead(dim=1024).to(device)
weights_path = hf_hub_download(repo_id="ozertuu/bge-m3-food-projection-head", filename="model.safetensors")
adapter.load_state_dict(load_file(weights_path))
adapter.eval()

# 3. Generate Culinary Embeddings
foods = [
    "Kuru Fasulye",
    "Soslu etli kuru fasulye yemeği",
    "Kuru fasulye konservesi",
    "Kuşbaşılı pide",
    "Kuşbaşı et"
]

with torch.inference_mode():
    base_embeddings = base_model.encode(foods, normalize_embeddings=True, convert_to_tensor=True, device=device)
    food_embeddings = adapter(base_embeddings)

# Cosine similarity
similarity_matrix = torch.matmul(food_embeddings, food_embeddings.T)
print("Similarity [Kuru Fasulye] & [Soslu Kuru Fasulye]:", round(similarity_matrix[0, 1].item(), 4))
print("Similarity [Kuru Fasulye] & [Konserve]:", round(similarity_matrix[0, 2].item(), 4))

---

📊 Training Details

• Loss Function: In-batch InfoNCE Contrastive Loss (Temperature $\tau = 0.05$).
• Dataset: 30,001 Anchor-Positive-HardNegative pairs (ozertuu/turkish-food-contrastive-triplets).
• Optimizer: AdamW (lr = 1e-3, weight_decay = 1e-4).
• Batch Size: 64.
• Epochs: 5.

---

📜 Citation

@misc{ozertuu2026bgem3food,
  author = {Ertugrul Ozer},
  title = {BGE-M3 Food Projection Head: Contrastive Culinary Domain Adapter},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {https://huggingface.co/ozertuu/bge-m3-food-projection-head}
}