Sriram-214/vyasalm-7b-gguf

🕉️ VyasaLM 7B GGUF

VyasaLM is a domain-adapted, spiritually intelligent language model fine-tuned on the complete collection of major Hindu scriptures. Built on top of Qwen2.5-7B-Instruct and optimized using Unsloth (4-bit QLoRA), the model is designed to answer complex theological, philosophical, and scriptural questions with reverence, accuracy, and deep contextual grounding.

🌟 Key Features

• Deep Scriptural Grounding: Fine-tuned on an extensive corpus of over 113 major scriptures including all four Vedas, Upanishads, Puranas, the Mahabharata, the Ramayana, and the Bhagavad Gita.
• Sanskrit Transliteration & Terminology: Native understanding of Sanskrit terms, slokas, transliterated concepts (like Dharma, Karma, Moksha), and philosophical commentaries.
• Reverent & Scholar-like Output: Programmed to answer questions with academic rigor, philosophical depth, and spiritual reverence.
• Quantized for Local Use: Available in Q4_K_M GGUF format, optimized for fast inference on consumer hardware (requires ~6GB VRAM/RAM).

---

📚 Training Corpus Details

The training dataset was parsed page-by-page from a curated library of 113 authoritative PDF volumes (approximately 72 MB of clean text) containing:

1. The Vedas: Rigveda, Yajurveda, Samaveda, and Atharvaveda.
2. The Upanishads: The 108 canonical Upanishads with primary commentaries.
3. The Epics (Itihasas): The complete Mahabharata (including the Bhagavad Gita) and the Ramayana.
4. The Puranas: Major Maha-Puranas (e.g., Vishnu Purana, Bhagavata Purana, Shiva Purana) and selected Upa-Puranas.

The text was preprocessed using a custom Sanskrit-aware cleaning pipeline to normalize OCR noise, segment sentences properly, and structure dialogue contexts.

---

⚡ Quick Start: Running Locally with Ollama

You can download and deploy the GGUF model locally using Ollama:

Step 1: Download the GGUF Model

Download Qwen2.5-7B-Instruct.Q4_K_M.gguf from this repository.

Step 2: Create a Modelfile

Create a file named Modelfile in the same directory as the downloaded GGUF file and add the following configuration:

FROM ./Qwen2.5-7B-Instruct.Q4_K_M.gguf

# Set prompt template (Qwen Chat template format)
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
"""

# Set custom system instructions
SYSTEM """You are VyasaLM, an elite Spiritual Foundation AI Model. Answer questions with reverence, accuracy, and deep scriptural context."""

# Set parameters
PARAMETER temperature 0.6
PARAMETER top_p 0.9

Step 3: Compile and Run in Ollama

Open your terminal and run:

# Create the Ollama model
ollama create vyasalm -f Modelfile

# Start a chat session
ollama run vyasalm "What is the difference between Advaita and Vishishtadvaita Vedanta?"

---

🐍 Usage in Python (Transformers / llama.cpp)

You can run the model directly in Python using llama-cpp-python:

from llama_cpp import Llama

# Load the model
llm = Llama(
    model_path="./Qwen2.5-7B-Instruct.Q4_K_M.gguf",
    n_ctx=2048,      # Context window size
    n_threads=4,     # Number of CPU threads
    n_gpu_layers=35  # Number of layers to offload to GPU
)

# Format the prompt
system_prompt = "You are VyasaLM, a spiritual foundation AI model."
user_prompt = "Explain the concept of Purusha and Prakriti according to Samkhya philosophy."

prompt = f"<|im_start|>system\n{system_prompt}<|im_end|>\n<|im_start|>user\n{user_prompt}<|im_end|>\n<|im_start|>assistant\n"

# Run inference
output = llm(prompt, max_tokens=512, stop=["<|im_end|>"])
print(output['choices'][0]['text'])

---

🛠️ Training Specifications

• Base Model: Qwen/Qwen2.5-7B-Instruct
• Method: 4-bit quantized LoRA (QLoRA)
• Framework: unsloth & TRL
• LoRA Hyperparameters: Rank $r=16$, Alpha $\alpha=16$, dropout $0$
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Sequence Length: 2048 tokens
• Optimizer: adamw_8bit
• Precision: Mixed precision (fp16/bf16 depending on hardware)

---

⚠️ Disclaimer

VyasaLM is designed for educational, research, and philosophical exploration. While it has been fine-tuned on authentic scriptures to avoid hallucinations, language models can occasionally output incorrect facts. Always consult original texts or verified traditional commentaries (Bhashyas) for scriptural study.

---

🤝 Acknowledgements

• Unsloth: For enabling memory-efficient 4-bit fine-tuning of 7B LLMs.
• Qwen Team: For providing the incredible Qwen2.5 base model.