PyTorch
GGUF
French
English
minimind
french
small-language-model
conversational

minimind-fr-pretrain-enfr

Bilingual en/fr base (next-token lm), ~64m, minimind arch. ~64 M params (hidden_size=768, num_hidden_layers=8, dense), MiniMind architecture. trained from scratch.

Round-1 prototype — fluent but small; treat outputs accordingly. Training framework: https://github.com/jingyaogong/minimind.

Files

file format
pretrain-enfr_768.pth raw PyTorch state_dict (fp16) — load with MiniMindForCausalLM(strict=False)
pretrain-enfr-f16.gguf GGUF F16 — exported via Qwen3ForCausalLM, runs in llama.cpp / Ollama / LM Studio
tokenizer.json, tokenizer_config.json byte-level BPE, vocab_size=6400, EN/FR

Config: num_attention_heads 8, num_key_value_heads 4, vocab_size 6400, max_position_embeddings 32768, rope_theta 1e6, tied embeddings, no MoE.

Training data

dataset contribution
yassinsiouda/minimind-fr-pretrain-enfr-data packaged training file for this model
allenai/c4 configs en + fr, ~2 GB raw text each, prose-filtered, split into <=1000-char chunks, shuffled

5,315,952 lines (2,549,541 fr / 2,766,411 en, ~48% French). The byte-level BPE-6400 tokenizer bundled here was retrained on this mix. Recipe: build_c4_enfr.py.

Run (GGUF)

llama-cli -m pretrain-enfr-f16.gguf -p "..." -ngl 99

Limitations

64 M parameters; English domain data for the specialists (French comes from the bilingual base, so domain idiom is anglicised); agentic capability is SFT-only (short 2-4 step tool loops); safety limited to the router's thin unsafe bucket.

License

Apache-2.0 (weights). Upstream dataset licenses govern downstream use — see the dataset repo.

Downloads last month
21
GGUF
Model size
63.9M params
Architecture
qwen3
Hardware compatibility
Log In to add your hardware

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for yassinsiouda/minimind-fr-pretrain-enfr

Finetunes
3 models

Datasets used to train yassinsiouda/minimind-fr-pretrain-enfr