empisto/phi4mini-coreflect-cbt-gguf

Empisto CoreReflect — Phi-4-mini CBT (Q4_K_M GGUF) · v3

A CBT-style reflective journaling companion small enough to run on a phone. A LoRA fine-tune of microsoft/Phi-4-mini-instruct, merged and quantized to Q4_K_M GGUF for llama.cpp.

Given a private journal entry, it responds with a short, grounded reflection: it names the thought or assumption without diagnosing, offers another reading, and leaves at most a small optional step.

File	phi4mini-cbt-v3-Q4_K_M.gguf (2.4 GB)
Base	microsoft/Phi-4-mini-instruct (phi3 arch, MIT)
Teacher	openai/gpt-oss-120b (Apache-2.0) — synthetic training data only
Adapter	LoRA r=16, α=32, on qkv_proj / o_proj / gate_up_proj / down_proj
Data	core_reflect_distill_v3 — 2,470 distilled CBT records (diverse/absent names)
Runtime	llama.cpp (Q4_K_M)

⚠️ You MUST send the system prompt

The model was trained with the CoreReflect persona in every example. Without it, quality drops sharply. Send this verbatim as the system message:

You are Empisto CoreReflect — a private, on-device journaling reflection companion.

Goal: help the user reflect safely and constructively using ONLY what they wrote.

Hard boundaries:
- Do NOT diagnose, label disorders, or provide medical/legal advice.
- Do NOT invent backstory, motives, trauma, or events not explicitly stated.
- Do NOT suggest extreme or high-stakes actions.
- Avoid strong imperatives like "you must", "you should", or "you need to".

Response style:
- Write 2 short paragraphs in natural language.
- Use plain everyday wording, not formal coaching or therapist-sounding phrasing.
- Ground the reply in one concrete detail from the user's writing within the first 1 to 2 sentences.
- Vary the opening and do not default to "From what you wrote, it sounds like..." or "It sounds like you're...".
- Stay close to the user's specifics and mention one concrete detail when it fits naturally.
- Reflect emotions, thoughts, and patterns gently before moving toward any next step.
- Most good replies should stay with reflection and grounded questions, not advice.
- Only suggest a tiny next step if the user is clearly asking what to do, and keep it to the final sentence.
- Ask 1 or 2 gentle clarifying questions that mention a concrete person, object, or situation from the user's writing when possible.
- Keep the opening sentences anchored to the actual people, objects, or events in the journal.
- Prefer one strong, specific question over two broad questions.
- Target roughly 95 to 165 words.
- Avoid stock phrases like "it might help to consider" or "a significant amount of mental fatigue".
- Avoid generic coping scripts like gratitude lists, "talk to someone you trust," or a default five-minute break unless the journal clearly points there.
- Avoid productivity tips, generic self-care tips, or broad coaching questions that could fit almost anyone.

Return ONLY the assistant reply text.

Usage

llama-server -m phi4mini-cbt-v3-Q4_K_M.gguf -c 2048

curl http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [
    {"role": "system", "content": "<the CoreReflect system prompt above>"},
    {"role": "user", "content": "I snapped at a colleague in standup over something trivial and I keep replaying it."}
  ],
  "temperature": 0.34, "max_tokens": 360
}'

Recommended: temperature 0.34, max_tokens 360 (the settings it was evaluated at).

Evaluation

Scored on held-out CBT prompts with a rule-based rubric (grounding, no diagnosis, no imperatives, no stock questions) plus human reading. hard = ambiguity, rumination, decision paralysis, subtle grounding failures.

model	canary	hard	name hallucination	safety
teacher gpt-oss-120b (ceiling)	10/10	8/10	0%	0
this model, Q4_K_M (v3)	10/10	8/10	0%	0
v2 (superseded — hallucinated names)	9/10	9/10	33%	0
base Phi-4-mini-instruct (floor)	0/10	0/10	—	0

v3 fixes a defect in v2: v2 invented a person ("Maya") into journals that named nobody, 33% of the time, because the training data over-used one name. v3 uses diverse names and a large no-name slice; name hallucination is 0% (measured on 48 samples of name-free journals).

• The base model scores 0/10 — it does not produce CoreReflect-style reflections at all. The gap is entirely learned.
• Quantization did not degrade quality. Q4_K_M is statistically indistinguishable from the unquantized adapter (the −1 canary / +2 hard is sampling noise at n=10).
• Zero safety-rule violations (no diagnosis language, no strong imperatives) across all evaluated replies.

Measured on an AMD Ryzen AI Max+ 395 (CPU): ~36 tok/s, ~4 GB resident, ~2 s cold start. These are desktop numbers, not phone numbers.

Intended use

Reflective journaling support inside the Empisto app. It responds to a single journal entry with a grounded reflection.

Limitations and risks

• This is not therapy, not a therapist, and not a medical device. It does not diagnose and must not be presented as clinical care.
• It has no crisis handling. It is not designed to detect or respond to self-harm, suicidality, or emergencies. Any product using it needs its own crisis-detection layer and escalation path.
• Evaluated at n=20 CBT prompts (name hallucination at n=48). A strong, consistent signal — but not a large-scale benchmark. Broader evaluation is warranted before high-stakes use.
• Not validated on a phone. On-device memory (2.5 GB + KV cache) is the likely constraint; real device numbers are unmeasured.
• English only, single-turn reflection. Trained on synthetic data (a teacher model's output), not real journals or clinical transcripts — it has never seen a real patient interaction.
• Small models fail in unpredictable ways. Human oversight is assumed.

Lineage and licensing

• Base weights: microsoft/Phi-4-mini-instruct — MIT. This model is a derivative; MIT applies.
• Training data: synthetic CBT records generated by openai/gpt-oss-120b (Apache-2.0). No real user journals were used.
• Teacher choice was made by measurement, not vendor claims — gpt-oss-120b beat Qwen3-32B on yield, reply quality, and speed in a head-to-head bakeoff.