Upload 12 files

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+gguf/HRM-Text-1B-Code-Tools-SFT-BF16.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/HRM-Text-1B-Code-Tools-SFT-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,15 @@
+Apache License 2.0
+
+Copyright 2026
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

README.md

@@ -1,3 +1,335 @@
 ---
+language:
+- en
 license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+base_model: sapientinc/HRM-Text-1B
+base_model_relation: finetune
+datasets:
+- pzarzycki/hrm-text-code-tools-sft
+inference: false
+tags:
+- hrm
+- hierarchical-reasoning
+- prefix-lm
+- code
+- tool-use
+- sft
+- transformers
+- safetensors
+- gguf
+- bf16
+- q8_0
+- llama.cpp
 ---
+
+# HRM-Text-1B Code and Tool-Use SFT
+
+This repository is a Transformers BF16 conversion of
+[`pzarzycki/hrm-text-1b-code-tools-sft`](https://huggingface.co/pzarzycki/hrm-text-1b-code-tools-sft),
+a full-parameter Stage A fine-tune of
+[`sapientinc/HRM-Text-1B`](https://huggingface.co/sapientinc/HRM-Text-1B).
+It also contains canonical BF16 and directly derived Q8_0 GGUF files.
+
+The source checkpoint is a research pilot trained for code generation and a
+fixed tool-transcript protocol. It has not undergone downstream benchmark or
+production-agent evaluation.
+
+## Model details
+
+| Field | Value |
+| --- | --- |
+| Architecture | HRM-Text PrefixLM |
+| Parameters | 1,182,795,264 |
+| Stored dtype | BF16 |
+| Hidden size | 1,536 |
+| H/L stack depth | 16 layers each |
+| Recurrence | `H_cycles=2`, `L_cycles=3` |
+| Attention | 12 heads, head dimension 128, gated MHA |
+| Intermediate size | 4,096, SwiGLU |
+| Context length | 4,096 tokens |
+| Vocabulary | 65,536 |
+| Position encoding | RoPE, theta 10,000 |
+| Normalization | Parameterless pre-RMSNorm |
+| Training stage | Stage A pilot, seed 17 |
+| Training budget | 10,000,147 response tokens |
+| Primary language | English |
+
+Stage B was not trained or published as part of the source revision used here.
+
+## Files
+
+| File | Format | Size | SHA256 |
+| --- | --- | ---: | --- |
+| `model.safetensors` | Transformers BF16 | 2,365,606,568 bytes | `2bc954894ab677dcdc66331863cf12aa4da8aa719fa56538a816632a3935546b` |
+| `gguf/HRM-Text-1B-Code-Tools-SFT-BF16.gguf` | GGUF BF16 | 2,367,996,448 bytes | `4c59b7f0187315ba942088c07e7aeee38339bbfdcc1da05f4e83d3efd3c22cf7` |
+| `gguf/HRM-Text-1B-Code-Tools-SFT-Q8_0.gguf` | GGUF Q8_0 | 1,259,127,360 bytes | `b0faf21c398ca4935d458d60decf6d3e60e9bc5d8dbfb1bf92d370b37bc9252e` |
+| `gguf/runtime/llama.cpp-hrm_text.patch` | llama.cpp runtime patch | - | See compatibility section |
+
+BF16 is the canonical storage format. Q8_0 was quantized directly from the
+BF16 GGUF. No F16 derivative is provided because converting BF16 to F16 would
+change 17,119 finite stored values and underflow 87 values to zero.
+
+## Requirements
+
+Use `transformers>=5.9.0`, which includes native `hrm_text` model support.
+The conversion and validation environment used Transformers 5.16.1 and
+PyTorch 2.13.0.
+
+```bash
+pip install --upgrade "transformers>=5.9.0" torch
+```
+
+Hosted inference is disabled in the model-card metadata because generic text
+generation endpoints do not provide the required PrefixLM `token_type_ids`.
+
+## Transformers usage
+
+The included Jinja template must be applied. It serializes the learned
+`direct` condition and the SFT transcript markup; this is not a Qwen/ChatML
+prompt despite using a Qwen-compatible tokenizer.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "YOUR_NAMESPACE/HRM-Text-1B-Code-Tools-SFT"
+device = torch.device(
+    "cuda" if torch.cuda.is_available()
+    else "mps" if torch.backends.mps.is_available()
+    else "cpu"
+)
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    dtype=torch.bfloat16,
+).to(device).eval()
+
+messages = [
+    {
+        "role": "user",
+        "content": "Write a Python function that returns the larger of two integers.",
+    }
+]
+inputs = tokenizer.apply_chat_template(
+    messages,
+    tokenize=True,
+    add_generation_prompt=True,
+    return_tensors="pt",
+    return_dict=True,
+).to(device)
+
+# HRM-Text was trained with a bidirectional prompt prefix.
+inputs["token_type_ids"] = torch.ones_like(inputs["input_ids"])
+
+with torch.inference_mode():
+    output_ids = model.generate(
+        **inputs,
+        max_new_tokens=256,
+        do_sample=False,
+    )
+
+new_ids = output_ids[0, inputs["input_ids"].shape[1]:]
+print(tokenizer.decode(new_ids, skip_special_tokens=False))
+```
+
+The rendered prompt starts with the following exact envelope:
+
+```text
+<|im_start|><|object_ref_start|><user>
+Write a Python function that returns the larger of two integers.
+</user>
+<assistant>
+<|im_end|>
+```
+
+Do not omit `token_type_ids` when using Transformers. A value of 1 marks a
+prompt position as part of the bidirectional prefix block. Omitting it falls
+back to pure-causal attention and does not match the training-time objective.
+
+### Tool schemas
+
+Pass OpenAI-style function schemas through the `tools` argument. The template
+places them inside the learned `<tools>...</tools>` transcript markup.
+
+```python
+tools = [
+    {
+        "type": "function",
+        "function": {
+            "name": "read_file",
+            "description": "Read a UTF-8 file relative to the task root.",
+            "parameters": {
+                "type": "object",
+                "properties": {"path": {"type": "string"}},
+                "required": ["path"],
+                "additionalProperties": False,
+            },
+        },
+    }
+]
+
+prompt = tokenizer.apply_chat_template(
+    [{"role": "user", "content": "Read README.md and summarize it."}],
+    tools=tools,
+    tokenize=False,
+    add_generation_prompt=True,
+)
+```
+
+`<user>`, `<assistant>`, `<tools>`, `<tool_call>`, and `<tool_result>` are
+ordinary learned text markup, not pretrained HRM control tokens. The model
+does not execute tools, validate arguments, or sandbox generated code. A
+`system` message is intentionally serialized with the same `<user>` markup;
+there is no separately trained system role.
+
+## GGUF compatibility
+
+The GGUF files use `general.architecture = hrm_text` and embed the exact Jinja
+template under `tokenizer.chat_template`. Standard unpatched llama.cpp,
+Ollama, LM Studio, and llama-cpp-python builds do not support this custom
+runtime graph at the time of this release.
+
+Apply the included patch to this exact llama.cpp commit:
+
+```text
+6a257d44633d4a752183ed778b88d2924d0a6b9d
+```
+
+```bash
+git clone https://github.com/ggml-org/llama.cpp
+cd llama.cpp
+git checkout 6a257d44633d4a752183ed778b88d2924d0a6b9d
+git apply /path/to/model/gguf/runtime/llama.cpp-hrm_text.patch
+cmake -B build -DGGML_METAL=ON -DGGML_NATIVE=OFF -DLLAMA_BUILD_UI=OFF
+cmake --build build --config Release --target llama-cli llama-server llama-quantize -j
+```
+
+Ninja is optional. The documented CMake flow works with the default Unix
+Makefiles generator, and Metal support is independent of the generator.
+
+Start the server with Jinja explicitly enabled. PrefixLM prefill must process
+the complete prompt in one physical batch, so set `--batch-size` and
+`--ubatch-size` to at least the maximum prompt length you intend to use. The
+example below supports prompts up to 512 tokens. Use `-ngl all` for Metal or
+`-ngl 0` for CPU-only inference.
+
+```bash
+./build/bin/llama-server \
+  -m /path/to/model/gguf/HRM-Text-1B-Code-Tools-SFT-Q8_0.gguf \
+  --alias HRM-Text-1B-Code-Tools-SFT \
+    --jinja --ctx-size 512 --batch-size 512 --ubatch-size 512 \
+    --cache-ram 0 --parallel 1 \
+    -ngl all --host 127.0.0.1 --port 8080
+```
+
+Requests to the OpenAI-compatible chat endpoint apply the embedded template:
+
+```bash
+curl http://127.0.0.1:8080/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "HRM-Text-1B-Code-Tools-SFT",
+    "messages": [{"role": "user", "content": "Write a Python max function."}],
+    "temperature": 0,
+        "max_tokens": 128,
+        "cache_prompt": false
+  }'
+```
+
+The patch maps `hrm_text.prefix_lm=true` to llama.cpp's non-causal attention
+mask. The complete initial prompt is therefore one bidirectional prefix block.
+Autoregressive token-by-token decoding remains causal in effect because the KV
+cache contains no future generated positions.
+
+This is deliberately narrower than arbitrary Transformers `token_type_ids`.
+Do not split one prefix across multiple physical batches, reuse a KV cache from
+a shorter prompt, or enable speculative multi-token decoding. Prompt-cache
+reuse is disabled in the command and request above. For the full 4,096-token
+context, set `--ctx-size`, `--batch-size`, and `--ubatch-size` to 4096 if the
+available memory permits it.
+
+## Conversion validation
+
+| Check | Result |
+| --- | --- |
+| Source Keras H5 to Transformers BF16 values | 1,182,795,264 checked, 0 bit mismatches |
+| Tensor mapping | 259 Keras tensors mapped to 131 fused Safetensors entries |
+| Keras vs Transformers FP32, pure causal | Maximum absolute logit difference `7.2718e-06` |
+| Keras vs Transformers FP32, direct PrefixLM | Maximum absolute logit difference `2.0981e-05` |
+| Author conversion tolerance | Passed at `atol=2e-4`, `rtol=2e-4` |
+| BF16 MPS top-1 | All tested positions matched |
+| BF16 MPS final-position top-10 | 10/10 overlap |
+| BF16 MPS two-step greedy generation | Matched for causal and PrefixLM cases |
+| Chat template | Plain and tool-schema cases passed |
+| GGUF structure | 259 tensors and embedded Jinja verified for BF16 and Q8_0 |
+| BF16 GGUF, Metal | 129/129 layers on MTL0; both two-token continuations matched Transformers PrefixLM baseline |
+| BF16 GGUF, CPU | 0/129 layers offloaded; both two-token continuations matched |
+| Q8_0 GGUF, Metal | 129/129 layers on MTL0; both two-token continuations matched |
+| llama.cpp PrefixLM mask | `prefix_lm=true`, `causal_attn=0`, complete-prompt prefill, prompt cache disabled |
+| llama.cpp Jinja | `/apply-template` strings and `/tokenize` IDs matched for plain and tool-schema prompts |
+| llama.cpp token ranking | Top-1 matched at all 12 checked steps; top-10 overlap was 9/10 or 10/10 |
+| OpenAI-compatible chat route | `/v1/chat/completions` passed with embedded Jinja enabled for all three runtime targets |
+
+BF16 logits are not bit-identical across Keras and Transformers because their
+RMSNorm, softmax, and backend arithmetic paths differ. Stored weights are
+bit-identical after the audited tensor mapping, FP32 outputs pass the source
+author's tolerance, and the tested BF16 token rankings and greedy outputs
+match.
+
+The llama.cpp comparisons use the same rendered token IDs and PrefixLM mask on
+both sides: every initial prompt token is bidirectional and generated tokens
+are causal. The validated two-step continuations were `[26763, 2336]` for the
+plain prompt and `[58, 19975]` for the tool-schema prompt on BF16 CPU, BF16
+Metal, and Q8_0 Metal.
+
+## Training provenance
+
+| Item | Value |
+| --- | --- |
+| SFT source repository | `pzarzycki/hrm-text-1b-code-tools-sft` |
+| SFT source revision | `ab083a772a29d4999251a437fe4e8b6007f828c3` |
+| Source Keras H5 SHA256 | `8ddb10cea110edff99380b360ec569e5dbde0d4a233d1c058b0e292700a42b84` |
+| Base model | `sapientinc/HRM-Text-1B` |
+| Training dataset | `pzarzycki/hrm-text-code-tools-sft`, canonical v2 Stage A |
+| Underlying data source | `nvidia/OpenCodeInstruct` revision `8f3ba5bafe4d6e8db46082cf7ae6741bc370604d` |
+| KerasHub reference implementation | `pzarzycki/keras-hub` revision `8e9207acfae1833c25ba6813932b4234b6b84bf9` |
+
+The Stage A selection contains 38,248 rows from the sealed training split.
+Training used full-parameter BF16 optimization with a 4,096-token context cap.
+See the source model card for the full optimizer setup and telemetry.
+
+## Intended use and limitations
+
+This checkpoint is intended for research on HRM-Text code adaptation,
+tool-call transcript generation, conversion fidelity, and local inference.
+
+- No downstream coding or agent benchmark has been reported for this pilot.
+- Training loss is not evidence of production coding-agent performance.
+- Generated code and tool calls may be incorrect, unsafe, or fabricated.
+- Tool execution, argument validation, permissions, and sandboxing must be
+  implemented by the host application.
+- The model is predominantly English and is limited to 4,096 tokens.
+- Evaluate task quality and safety independently before deployment.
+
+## License and citation
+
+The model is released under the Apache License 2.0. The Stage A dataset traces
+to CC-BY-4.0 data; consult the linked dataset card for its attribution and
+usage terms.
+
+Please cite the base HRM-Text work:
+
+```bibtex
+@misc{wang2026hrmtextefficientpretrainingscaling,
+  title={HRM-Text: Efficient Pretraining Beyond Scaling},
+  author={Guan Wang and Changling Liu and Chenyu Wang and Cai Zhou and Yuhao Sun and Yifei Wu and Shuai Zhen and Luca Scimeca and Yasin Abbasi Yadkori},
+  year={2026},
+  eprint={2605.20613},
+  archivePrefix={arXiv},
+  primaryClass={cs.CL},
+  url={https://arxiv.org/abs/2605.20613}
+}
+```

chat_template.jinja

@@ -0,0 +1,32 @@
+{{- '<|im_start|><|object_ref_start|>' -}}
+{%- if tools -%}
+{{- '<tools>' -}}
+{{- tools | tojson -}}
+{{- '</tools>\n' -}}
+{%- endif -%}
+{%- for message in messages -%}
+{%- if message['role'] == 'system' -%}
+{{- '<user>\n' + message['content'] + '\n</user>\n' -}}
+{%- elif message['role'] == 'user' -%}
+{{- '<user>\n' + message['content'] + '\n</user>\n' -}}
+{%- elif message['role'] == 'assistant' -%}
+{{- '<assistant>\n' -}}
+{%- if message.get('content') -%}
+{{- message['content'] -}}
+{%- endif -%}
+{%- for tool_call in message.get('tool_calls', []) -%}
+{{- '<tool_call>' -}}
+{{- {'arguments': tool_call['function']['arguments'], 'name': tool_call['function']['name']} | tojson -}}
+{{- '</tool_call>' -}}
+{%- endfor -%}
+{{- '\n' -}}
+{%- elif message['role'] == 'tool' -%}
+{{- '<tool_result>' + message['content'] + '</tool_result>\n' -}}
+{%- else -%}
+{{- raise_exception('Unsupported message role: ' + message['role']) -}}
+{%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+{{- '<assistant>\n' -}}
+{%- endif -%}
+{{- '<|im_end|>' -}}

config.json

@@ -0,0 +1,31 @@
+{
+  "model_type": "hrm_text",
+  "architectures": [
+    "HrmTextForCausalLM"
+  ],
+  "vocab_size": 65536,
+  "hidden_size": 1536,
+  "intermediate_size": 4096,
+  "num_hidden_layers": 16,
+  "num_attention_heads": 12,
+  "num_key_value_heads": 12,
+  "head_dim": 128,
+  "H_cycles": 2,
+  "L_cycles": 3,
+  "L_bp_cycles": [
+    0,
+    3
+  ],
+  "max_position_embeddings": 4096,
+  "rms_norm_eps": 1e-06,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": false,
+  "initializer_range": 0.025515518153991442,
+  "embedding_scale": 39.191835884530846,
+  "prefix_lm": true,
+  "pad_token_id": 5,
+  "bos_token_id": 6,
+  "eos_token_id": 11,
+  "dtype": "bfloat16",
+  "transformers_version": "5.16.1"
+}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 6,
+  "eos_token_id": 11,
+  "pad_token_id": 5,
+  "transformers_version": "5.16.1"
+}

gguf/HRM-Text-1B-Code-Tools-SFT-BF16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c59b7f0187315ba942088c07e7aeee38339bbfdcc1da05f4e83d3efd3c22cf7
+size 2367996448

gguf/HRM-Text-1B-Code-Tools-SFT-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0faf21c398ca4935d458d60decf6d3e60e9bc5d8dbfb1bf92d370b37bc9252e
+size 1259127360

gguf/runtime/llama.cpp-hrm_text.patch

@@ -0,0 +1,572 @@
+diff --git a/conversion/__init__.py b/conversion/__init__.py
+index 2c38123df..ecf1be2db 100644
+--- a/conversion/__init__.py
++++ b/conversion/__init__.py
+@@ -95,6 +95,7 @@ TEXT_MODEL_MAP: dict[str, str] = {
+     "HunYuanDenseV1ForCausalLM": "hunyuan",
+     "HunYuanMoEV1ForCausalLM": "hunyuan",
+     "HunYuanVLForConditionalGeneration": "hunyuan",
++    "HrmTextForCausalLM": "hrm_text",
+     "IQuestCoderForCausalLM": "llama",
+     "InternLM2ForCausalLM": "internlm",
+     "InternLM3ForCausalLM": "internlm",
+diff --git a/conversion/hrm_text.py b/conversion/hrm_text.py
+new file mode 100644
+index 000000000..1f29ab55e
+--- /dev/null
++++ b/conversion/hrm_text.py
+@@ -0,0 +1,120 @@
++from __future__ import annotations
++
++import re
++import json
++
++from typing import Iterable, TYPE_CHECKING
++
++import torch
++
++if TYPE_CHECKING:
++    from torch import Tensor
++
++from .base import ModelBase, TextModel, gguf, logger
++
++
++@ModelBase.register("HrmTextForCausalLM")
++class HrmTextModel(TextModel):
++    model_arch = gguf.MODEL_ARCH.HRM_TEXT
++
++    def __init__(self, *args, **kwargs):
++        super().__init__(*args, **kwargs)
++
++        with open(self.dir_model / "config.json", "r", encoding="utf-8") as f:
++            self.raw_hparams = json.load(f)
++
++        self.layers_per_stack = self.raw_hparams["num_hidden_layers"]
++        self.h_cycles = self.raw_hparams["H_cycles"]
++        self.l_cycles = self.raw_hparams["L_cycles"]
++        self.physical_block_count = self.layers_per_stack * 2
++        self.cache_block_count = self.layers_per_stack * self.h_cycles * (self.l_cycles + 1)
++
++        # GGUF tensors store one physical L stack followed by one physical H stack.
++        # The runtime expands these 32 physical layers across 128 KV-cache slots.
++        self.block_count = self.physical_block_count
++        self.tensor_map = gguf.get_tensor_name_map(self.model_arch, self.block_count)
++
++    def set_vocab(self):
++        # HRM-Text ships a Qwen2-style tokenizer.json. Keep it as a plain tokenizer;
++        # do not add a chat template for validation GGUFs.
++        self._set_vocab_gpt2()
++
++    def get_vocab_base_pre(self, tokenizer) -> str:
++        del tokenizer
++        return "qwen2"
++
++    def set_gguf_parameters(self):
++        hp = self.raw_hparams
++        head_dim = hp["head_dim"]
++
++        self.gguf_writer.add_context_length(hp["max_position_embeddings"])
++        self.gguf_writer.add_embedding_length(hp["hidden_size"])
++        self.gguf_writer.add_block_count(self.cache_block_count)
++        self.gguf_writer.add_feed_forward_length(hp["intermediate_size"])
++        self.gguf_writer.add_head_count(hp["num_attention_heads"])
++        self.gguf_writer.add_head_count_kv(hp["num_key_value_heads"])
++        self.gguf_writer.add_key_length(head_dim)
++        self.gguf_writer.add_value_length(head_dim)
++        self.gguf_writer.add_rope_dimension_count(head_dim)
++        self.gguf_writer.add_rope_freq_base(hp.get("rope_theta", 10000.0))
++        self.gguf_writer.add_layer_norm_rms_eps(hp["rms_norm_eps"])
++        self.gguf_writer.add_embedding_scale(hp["embedding_scale"])
++
++        arch = self.gguf_writer.arch
++        self.gguf_writer.add_uint32(gguf.Keys.LLM.HRM_LAYERS_PER_STACK.format(arch=arch), self.layers_per_stack)
++        self.gguf_writer.add_uint32(gguf.Keys.LLM.HRM_H_CYCLES.format(arch=arch), self.h_cycles)
++        self.gguf_writer.add_uint32(gguf.Keys.LLM.HRM_L_CYCLES.format(arch=arch), self.l_cycles)
++        self.gguf_writer.add_bool(gguf.Keys.LLM.HRM_PREFIX_LM.format(arch=arch), bool(hp.get("prefix_lm", False)))
++
++    def _format(self, key: gguf.MODEL_TENSOR, bid: int | None = None, suffix: str = ".weight") -> str:
++        return self.format_tensor_name(key, bid=bid, suffix=suffix)
++
++    def modify_tensors(self, data_torch: Tensor, name: str, bid: int | None) -> Iterable[tuple[str, Tensor]]:
++        if name == "model.embed_tokens.weight":
++            yield self._format(gguf.MODEL_TENSOR.TOKEN_EMBD), data_torch
++            return
++
++        if name == "lm_head.weight":
++            yield self._format(gguf.MODEL_TENSOR.OUTPUT), data_torch
++            return
++
++        if name == "model.z_L_init":
++            yield self._format(gguf.MODEL_TENSOR.HRM_Z_L_INIT, suffix=""), data_torch
++            return
++
++        match = re.fullmatch(r"model\.([LH])_module\.layers\.(\d+)\.(.+)", name)
++        if match is None:
++            raise ValueError(f"Can not map tensor {name!r}")
++
++        stack, layer_s, tensor_name = match.groups()
++        layer_idx = int(layer_s)
++        if layer_idx >= self.layers_per_stack:
++            raise ValueError(f"Layer index {layer_idx} outside HRM stack size {self.layers_per_stack}")
++
++        physical_bid = layer_idx + (self.layers_per_stack if stack == "H" else 0)
++
++        if tensor_name == "attn.gqkv_proj.weight":
++            gate, q, k, v = torch.chunk(data_torch, 4, dim=0)
++            logger.debug("Split %s as gate, q, k, v", name)
++            yield self._format(gguf.MODEL_TENSOR.ATTN_GATE, physical_bid), gate.contiguous()
++            yield self._format(gguf.MODEL_TENSOR.ATTN_Q, physical_bid), q.contiguous()
++            yield self._format(gguf.MODEL_TENSOR.ATTN_K, physical_bid), k.contiguous()
++            yield self._format(gguf.MODEL_TENSOR.ATTN_V, physical_bid), v.contiguous()
++            return
++
++        if tensor_name == "attn.o_proj.weight":
++            yield self._format(gguf.MODEL_TENSOR.ATTN_OUT, physical_bid), data_torch
++            return
++
++        if tensor_name == "mlp.gate_up_proj.weight":
++            gate, up = torch.chunk(data_torch, 2, dim=0)
++            logger.debug("Split %s as gate, up", name)
++            yield self._format(gguf.MODEL_TENSOR.FFN_GATE, physical_bid), gate.contiguous()
++            yield self._format(gguf.MODEL_TENSOR.FFN_UP, physical_bid), up.contiguous()
++            return
++
++        if tensor_name == "mlp.down_proj.weight":
++            yield self._format(gguf.MODEL_TENSOR.FFN_DOWN, physical_bid), data_torch
++            return
++
++        raise ValueError(f"Can not map tensor {name!r}")
+diff --git a/gguf-py/gguf/constants.py b/gguf-py/gguf/constants.py
+index 7fdcf03d7..b84cc8827 100644
+--- a/gguf-py/gguf/constants.py
++++ b/gguf-py/gguf/constants.py
+@@ -144,6 +144,10 @@ class Keys:
+         TOKEN_SHIFT_COUNT                 = "{arch}.token_shift_count"
+         INTERLEAVE_MOE_LAYER_STEP         = "{arch}.interleave_moe_layer_step"
+         FULL_ATTENTION_INTERVAL           = "{arch}.full_attention_interval"
++        HRM_LAYERS_PER_STACK              = "{arch}.layers_per_stack"
++        HRM_H_CYCLES                      = "{arch}.h_cycles"
++        HRM_L_CYCLES                      = "{arch}.l_cycles"
++        HRM_PREFIX_LM                     = "{arch}.prefix_lm"
+         ACTIVATION_SPARSITY_SCALE         = "{arch}.activation_sparsity_scale"
+         ALTUP_ACTIVE_IDX                  = "{arch}.altup.active_idx"
+         ALTUP_NUM_INPUTS                  = "{arch}.altup.num_inputs"
+@@ -410,6 +414,7 @@ class MODEL_ARCH(IntEnum):
+     QWEN3            = auto()
+     QWEN3MOE         = auto()
+     QWEN3NEXT        = auto()
++    HRM_TEXT         = auto()
+     QWEN3VL          = auto()
+     QWEN3VLMOE       = auto()
+     QWEN35           = auto()
+@@ -527,6 +532,7 @@ class MODEL_TENSOR(IntEnum):
+     TOKEN_TYPES          = auto()
+     POS_EMBD             = auto()
+     OUTPUT               = auto()
++    HRM_Z_L_INIT         = auto()
+     DENSE_2_OUT          = auto() # embeddinggemma 2_Dense
+     DENSE_3_OUT          = auto() # embeddinggemma 3_Dense
+     OUTPUT_NORM          = auto()
+@@ -925,6 +931,7 @@ MODEL_ARCH_NAMES: dict[MODEL_ARCH, str] = {
+     MODEL_ARCH.QWEN3:            "qwen3",
+     MODEL_ARCH.QWEN3MOE:         "qwen3moe",
+     MODEL_ARCH.QWEN3NEXT:        "qwen3next",
++    MODEL_ARCH.HRM_TEXT:         "hrm_text",
+     MODEL_ARCH.QWEN3VL:          "qwen3vl",
+     MODEL_ARCH.QWEN3VLMOE:       "qwen3vlmoe",
+     MODEL_ARCH.QWEN35:           "qwen35",
+@@ -1042,6 +1049,7 @@ TENSOR_NAMES: dict[MODEL_TENSOR, str] = {
+     MODEL_TENSOR.POS_EMBD:                  "position_embd",
+     MODEL_TENSOR.OUTPUT_NORM:               "output_norm",
+     MODEL_TENSOR.OUTPUT:                    "output",
++    MODEL_TENSOR.HRM_Z_L_INIT:              "hrm.z_l_init",
+     MODEL_TENSOR.DENSE_2_OUT:                "dense_2", # embeddinggemma 2_Dense
+     MODEL_TENSOR.DENSE_3_OUT:                "dense_3", # embeddinggemma 2_Dense
+     MODEL_TENSOR.ROPE_FREQS:                "rope_freqs",
+@@ -2057,6 +2065,19 @@ MODEL_TENSORS: dict[MODEL_ARCH, list[MODEL_TENSOR]] = {
+         MODEL_TENSOR.SSM_BETA_ALPHA,
+         MODEL_TENSOR.SSM_OUT
+     ],
++    MODEL_ARCH.HRM_TEXT: [
++        MODEL_TENSOR.TOKEN_EMBD,
++        MODEL_TENSOR.OUTPUT,
++        MODEL_TENSOR.HRM_Z_L_INIT,
++        MODEL_TENSOR.ATTN_Q,
++        MODEL_TENSOR.ATTN_K,
++        MODEL_TENSOR.ATTN_V,
++        MODEL_TENSOR.ATTN_GATE,
++        MODEL_TENSOR.ATTN_OUT,
++        MODEL_TENSOR.FFN_GATE,
++        MODEL_TENSOR.FFN_DOWN,
++        MODEL_TENSOR.FFN_UP,
++    ],
+     MODEL_ARCH.QWEN3VL: [
+         MODEL_TENSOR.TOKEN_EMBD,
+         MODEL_TENSOR.OUTPUT_NORM,
+diff --git a/src/llama-arch.cpp b/src/llama-arch.cpp
+index c9eead18a..5b8ee3781 100644
+--- a/src/llama-arch.cpp
++++ b/src/llama-arch.cpp
+@@ -37,6 +37,7 @@ static const std::map<llm_arch, const char *> LLM_ARCH_NAMES = {
+     { LLM_ARCH_QWEN3,            "qwen3"            },
+     { LLM_ARCH_QWEN3MOE,         "qwen3moe"         },
+     { LLM_ARCH_QWEN3NEXT,        "qwen3next"        },
++    { LLM_ARCH_HRM_TEXT,         "hrm_text"         },
+     { LLM_ARCH_QWEN3VL,          "qwen3vl"          },
+     { LLM_ARCH_QWEN3VLMOE,       "qwen3vlmoe"       },
+     { LLM_ARCH_QWEN35,           "qwen35"           },
+@@ -209,6 +210,10 @@ static const std::map<llm_kv, const char *> LLM_KV_NAMES = {
+     { LLM_KV_TOKEN_SHIFT_COUNT,                 "%s.token_shift_count"                 },
+     { LLM_KV_INTERLEAVE_MOE_LAYER_STEP,         "%s.interleave_moe_layer_step"         },
+     { LLM_KV_FULL_ATTENTION_INTERVAL,           "%s.full_attention_interval"           },
++    { LLM_KV_HRM_LAYERS_PER_STACK,              "%s.layers_per_stack"                  },
++    { LLM_KV_HRM_H_CYCLES,                      "%s.h_cycles"                          },
++    { LLM_KV_HRM_L_CYCLES,                      "%s.l_cycles"                          },
++    { LLM_KV_HRM_PREFIX_LM,                     "%s.prefix_lm"                         },
+ 
+     { LLM_KV_ATTENTION_HEAD_COUNT,                   "%s.attention.head_count"                   },
+     { LLM_KV_ATTENTION_HEAD_COUNT_KV,                "%s.attention.head_count_kv"                },
+@@ -346,6 +351,7 @@ static const std::map<llm_tensor, const char *> LLM_TENSOR_NAMES = {
+     { LLM_TENSOR_OUTPUT_NORM,                            "output_norm" },
+     { LLM_TENSOR_OUTPUT_NORM_LFM2,                       "token_embd_norm" }, // fix for wrong tensor name
+     { LLM_TENSOR_OUTPUT,                                 "output" },
++    { LLM_TENSOR_HRM_Z_L_INIT,                           "hrm.z_l_init" },
+     { LLM_TENSOR_ROPE_FREQS,                             "rope_freqs" },
+     { LLM_TENSOR_ATTN_NORM,                              "blk.%d.attn_norm" },
+     { LLM_TENSOR_ATTN_Q,                                 "blk.%d.attn_q" },
+@@ -565,6 +571,7 @@ static const std::map<llm_tensor, llm_tensor_info> LLM_TENSOR_INFOS = {
+     {LLM_TENSOR_POS_EMBD,                   {LLM_TENSOR_LAYER_INPUT,     GGML_OP_GET_ROWS}},
+     {LLM_TENSOR_TOKEN_TYPES,                {LLM_TENSOR_LAYER_INPUT,     GGML_OP_GET_ROWS}},
+     {LLM_TENSOR_TOKEN_EMBD_NORM,            {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},  // do the norms on the first layer (not the input layer)
++    {LLM_TENSOR_HRM_Z_L_INIT,               {LLM_TENSOR_LAYER_INPUT,     GGML_OP_MUL}},
+     {LLM_TENSOR_OUTPUT,                     {LLM_TENSOR_LAYER_OUTPUT,    GGML_OP_MUL_MAT}},
+     {LLM_TENSOR_CLS,                        {LLM_TENSOR_LAYER_OUTPUT,    GGML_OP_MUL_MAT}},
+     {LLM_TENSOR_CLS_OUT,                    {LLM_TENSOR_LAYER_OUTPUT,    GGML_OP_MUL_MAT}},
+diff --git a/src/llama-arch.h b/src/llama-arch.h
+index 89cf16cc3..fa04b684b 100644
+--- a/src/llama-arch.h
++++ b/src/llama-arch.h
+@@ -41,6 +41,7 @@ enum llm_arch {
+     LLM_ARCH_QWEN3,
+     LLM_ARCH_QWEN3MOE,
+     LLM_ARCH_QWEN3NEXT,
++    LLM_ARCH_HRM_TEXT,
+     LLM_ARCH_QWEN3VL,
+     LLM_ARCH_QWEN3VLMOE,
+     LLM_ARCH_QWEN35,
+@@ -213,6 +214,10 @@ enum llm_kv {
+     LLM_KV_TOKEN_SHIFT_COUNT,
+     LLM_KV_INTERLEAVE_MOE_LAYER_STEP,
+     LLM_KV_FULL_ATTENTION_INTERVAL,
++    LLM_KV_HRM_LAYERS_PER_STACK,
++    LLM_KV_HRM_H_CYCLES,
++    LLM_KV_HRM_L_CYCLES,
++    LLM_KV_HRM_PREFIX_LM,
+ 
+     LLM_KV_ATTENTION_HEAD_COUNT,
+     LLM_KV_ATTENTION_HEAD_COUNT_KV,
+@@ -354,6 +359,7 @@ enum llm_tensor {
+     LLM_TENSOR_DENSE_2_OUT,
+     LLM_TENSOR_DENSE_3_OUT,
+     LLM_TENSOR_OUTPUT,
++    LLM_TENSOR_HRM_Z_L_INIT,
+     LLM_TENSOR_OUTPUT_NORM,
+     LLM_TENSOR_OUTPUT_NORM_LFM2, // fix for wrong tensor name
+     LLM_TENSOR_ROPE_FREQS,
+diff --git a/src/llama-context.cpp b/src/llama-context.cpp
+index ad36c0666..fa80f4260 100644
+--- a/src/llama-context.cpp
++++ b/src/llama-context.cpp
+@@ -2208,6 +2208,9 @@ uint32_t llama_context::graph_max_nodes(uint32_t n_tokens) const {
+     if (model.arch == LLM_ARCH_QWEN3NEXT || model.arch == LLM_ARCH_KIMI_LINEAR || model.arch == LLM_ARCH_QWEN35 || model.arch == LLM_ARCH_QWEN35MOE) {
+         return std::max<uint32_t>(n_tokens * 40, 32u * model.n_tensors());
+     }
++    if (model.arch == LLM_ARCH_HRM_TEXT) {
++        return std::max<uint32_t>(n_tokens * 80, 64u * model.n_tensors());
++    }
+     uint32_t res = std::max<uint32_t>(1024u, 8u*model.n_tensors());
+     for (const auto & lora : model.loras) {
+         res += lora->get_n_nodes();
+diff --git a/src/llama-hparams.h b/src/llama-hparams.h
+index e2d051edc..812598f69 100644
+--- a/src/llama-hparams.h
++++ b/src/llama-hparams.h
+@@ -164,6 +164,12 @@ struct llama_hparams {
+     float f_embedding_scale = 0.0f;
+     float f_attention_scale = 0.0f;
+ 
++    // HRM-Text recurrence metadata. n_layer remains the expanded KV-cache slot count.
++    uint32_t n_hrm_layer_per_stack = 0;
++    uint32_t n_hrm_h_cycles        = 0;
++    uint32_t n_hrm_l_cycles        = 0;
++    bool     hrm_prefix_lm         = false;
++
+     // grok-2
+     float    f_attn_out_scale = 0.0f;
+     uint32_t attn_temp_length = 0;
+diff --git a/src/llama-model-saver.cpp b/src/llama-model-saver.cpp
+index 528e4c9c0..8a6e009c6 100644
+--- a/src/llama-model-saver.cpp
++++ b/src/llama-model-saver.cpp
+@@ -245,6 +245,10 @@ void llama_model_saver::add_kv_from_model() {
+     add_kv(LLM_KV_TOKEN_SHIFT_COUNT,                 hparams.token_shift_count);
+     add_kv(LLM_KV_INTERLEAVE_MOE_LAYER_STEP,         hparams.n_moe_layer_step);
+     // add_kv(LLM_KV_FULL_ATTENTION_INTERVAL,           ???);
++    add_kv(LLM_KV_HRM_LAYERS_PER_STACK,              hparams.n_hrm_layer_per_stack);
++    add_kv(LLM_KV_HRM_H_CYCLES,                      hparams.n_hrm_h_cycles);
++    add_kv(LLM_KV_HRM_L_CYCLES,                      hparams.n_hrm_l_cycles);
++    add_kv(LLM_KV_HRM_PREFIX_LM,                     hparams.hrm_prefix_lm);
+ 
+     add_kv(LLM_KV_ATTENTION_HEAD_COUNT,              hparams.n_head_arr, true);
+     add_kv(LLM_KV_ATTENTION_HEAD_COUNT_KV,           hparams.n_head_kv_arr, true);
+diff --git a/src/llama-model.cpp b/src/llama-model.cpp
+index 8bf20a716..a3cc996aa 100644
+--- a/src/llama-model.cpp
++++ b/src/llama-model.cpp
+@@ -96,6 +96,8 @@ static llama_model * llama_model_mapping(llm_arch arch, const llama_model_params
+             return new llama_model_qwen2moe(params);
+         case LLM_ARCH_QWEN3:
+             return new llama_model_qwen3(params);
++        case LLM_ARCH_HRM_TEXT:
++            return new llama_model_hrm_text(params);
+         case LLM_ARCH_QWEN3MOE:
+             return new llama_model_qwen3moe(params);
+         case LLM_ARCH_QWEN3VL:
+@@ -2339,6 +2341,7 @@ llama_rope_type llama_model_rope_type(const llama_model * model) {
+         case LLM_ARCH_PANGU_EMBED:
+         case LLM_ARCH_AFMOE:
+         case LLM_ARCH_QWEN3NEXT:
++        case LLM_ARCH_HRM_TEXT:
+         case LLM_ARCH_MIMO2:
+         case LLM_ARCH_STEP35:
+             return LLAMA_ROPE_TYPE_NEOX;
+diff --git a/src/models/hrm-text.cpp b/src/models/hrm-text.cpp
+new file mode 100644
+index 000000000..e0a3e9f59
+--- /dev/null
++++ b/src/models/hrm-text.cpp
+@@ -0,0 +1,184 @@
++#include "models.h"
++
++#include <cmath>
++#include <vector>
++
++void llama_model_hrm_text::load_arch_hparams(llama_model_loader & ml) {
++    ml.get_key(LLM_KV_ATTENTION_LAYERNORM_RMS_EPS, hparams.f_norm_rms_eps);
++    ml.get_key(LLM_KV_EMBEDDING_SCALE,             hparams.f_embedding_scale);
++    ml.get_key(LLM_KV_HRM_LAYERS_PER_STACK,        hparams.n_hrm_layer_per_stack);
++    ml.get_key(LLM_KV_HRM_H_CYCLES,                hparams.n_hrm_h_cycles);
++    ml.get_key(LLM_KV_HRM_L_CYCLES,                hparams.n_hrm_l_cycles);
++    ml.get_key(LLM_KV_HRM_PREFIX_LM,               hparams.hrm_prefix_lm, false);
++    hparams.causal_attn = !hparams.hrm_prefix_lm;
++
++    switch (hparams.n_embd) {
++        case 1536: type = LLM_TYPE_1B; break;
++        default:   type = LLM_TYPE_UNKNOWN;
++    }
++}
++
++void llama_model_hrm_text::load_arch_tensors(llama_model_loader &) {
++    LLAMA_LOAD_LOCALS;
++
++    const int64_t n_stack = hparams.n_hrm_layer_per_stack;
++    const int64_t n_cycle_slots = n_stack * (hparams.n_hrm_l_cycles + 1);
++
++    tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
++    output   = create_tensor(tn(LLM_TENSOR_OUTPUT,     "weight"), {n_embd, n_vocab}, 0);
++
++    hrm_z_l_init = create_tensor(tn(LLM_TENSOR_HRM_Z_L_INIT), {n_embd}, 0);
++
++    std::vector<bool> loaded_physical(2 * n_stack, false);
++
++    for (int il = 0; il < n_layer; ++il) {
++        auto & layer = layers[il];
++
++        const int64_t layer_in_stack = il % n_stack;
++        const int64_t phase = (il % n_cycle_slots) / n_stack;
++        const bool is_h_stack = phase == int64_t(hparams.n_hrm_l_cycles);
++        const int physical_bid = int((is_h_stack ? n_stack : 0) + layer_in_stack);
++
++        const int flags = loaded_physical[physical_bid] ? TENSOR_DUPLICATED : 0;
++        loaded_physical[physical_bid] = true;
++
++        create_tensor_qkv(layer, physical_bid,
++                n_embd,
++                n_embd_head_k * n_head,
++                n_embd_k_gqa,
++                n_embd_v_gqa,
++                flags);
++
++        layer.wqkv_gate = create_tensor(tn(LLM_TENSOR_ATTN_GATE, "weight", physical_bid), {n_embd, n_embd_head_k * n_head}, flags);
++        layer.wo        = create_tensor(tn(LLM_TENSOR_ATTN_OUT,  "weight", physical_bid), {n_embd_head_k * n_head, n_embd}, flags);
++
++        layer.ffn_gate = create_tensor(tn(LLM_TENSOR_FFN_GATE, "weight", physical_bid), {n_embd, n_ff}, flags);
++        layer.ffn_down = create_tensor(tn(LLM_TENSOR_FFN_DOWN, "weight", physical_bid), {n_ff, n_embd}, flags);
++        layer.ffn_up   = create_tensor(tn(LLM_TENSOR_FFN_UP,   "weight", physical_bid), {n_embd, n_ff}, flags);
++    }
++}
++
++std::unique_ptr<llm_graph_context> llama_model_hrm_text::build_arch_graph(const llm_graph_params & params) const {
++    return std::make_unique<graph>(*this, params);
++}
++
++llama_model_hrm_text::graph::graph(const llama_model & model_, const llm_graph_params & params) : llm_graph_context(params) {
++    const auto & model = static_cast<const llama_model_hrm_text &>(model_);
++
++    GGML_ASSERT(model.tok_embd != nullptr);
++    GGML_ASSERT(model.output != nullptr);
++    GGML_ASSERT(model.hrm_z_l_init != nullptr);
++
++    const int64_t n_embd_head = hparams.n_embd_head_v();
++    GGML_ASSERT(n_embd_head == hparams.n_embd_head_k());
++    GGML_ASSERT(n_embd_head == n_rot);
++
++    const int64_t n_stack = hparams.n_hrm_layer_per_stack;
++    const int64_t h_cycles = hparams.n_hrm_h_cycles;
++    const int64_t l_cycles = hparams.n_hrm_l_cycles;
++
++    ggml_tensor * inp_pos = build_inp_pos();
++    auto * inp_attn = build_attn_inp_kv();
++    ggml_tensor * inp_out_ids = build_inp_out_ids();
++
++    ggml_tensor * hidden_high = build_inp_embd(model.tok_embd);
++    ggml_tensor * hidden_low = ggml_repeat(ctx0, model.hrm_z_l_init, hidden_high);
++    cb(hidden_low, "hrm_z_l_init", -1);
++
++    const float kq_scale = 1.0f / std::sqrt(float(n_embd_head));
++
++    auto build_stack = [&](ggml_tensor * stack_inp, int slot_offset) -> ggml_tensor * {
++        ggml_tensor * stack_cur = stack_inp;
++
++        for (int layer_idx = 0; layer_idx < n_stack; ++layer_idx) {
++            const int il = slot_offset + layer_idx;
++            const auto & layer = model.layers[il];
++
++            ggml_tensor * inpSA = stack_cur;
++            ggml_tensor * cur = build_norm(stack_cur, nullptr, nullptr, LLM_NORM_RMS, il);
++            cb(cur, "attn_norm", il);
++
++            {
++                ggml_tensor * attn_inp = cur;
++                auto [Qcur, Kcur, Vcur] = build_qkv(layer, cur, n_embd_head, n_head, n_head_kv, il);
++
++                ggml_tensor * gate = build_lora_mm(layer.wqkv_gate, attn_inp, layer.wqkv_gate_s);
++                cb(gate, "attn_gate_proj", il);
++
++                Qcur = ggml_rope_ext(
++                        ctx0, Qcur, inp_pos, nullptr,
++                        n_rot, rope_type, n_ctx_orig, freq_base, freq_scale,
++                        ext_factor, attn_factor, beta_fast, beta_slow);
++                cb(Qcur, "Qcur_rope", il);
++
++                Kcur = ggml_rope_ext(
++                        ctx0, Kcur, inp_pos, nullptr,
++                        n_rot, rope_type, n_ctx_orig, freq_base, freq_scale,
++                        ext_factor, attn_factor, beta_fast, beta_slow);
++                cb(Kcur, "Kcur_rope", il);
++
++                cur = build_attn(inp_attn,
++                        nullptr, nullptr, nullptr,
++                        Qcur, Kcur, Vcur, nullptr, nullptr, nullptr, kq_scale, il);
++                cb(cur, "attn_out", il);
++
++                gate = ggml_sigmoid(ctx0, gate);
++                cb(gate, "attn_gate_sig", il);
++
++                cur = ggml_mul(ctx0, cur, gate);
++                cb(cur, "attn_gated", il);
++
++                cur = build_lora_mm(layer.wo, cur, layer.wo_s);
++                cb(cur, "attn_o_proj", il);
++            }
++
++            ggml_tensor * ffn_inp = ggml_add(ctx0, cur, inpSA);
++            cb(ffn_inp, "ffn_inp", il);
++
++            cur = build_norm(ffn_inp, nullptr, nullptr, LLM_NORM_RMS, il);
++            cb(cur, "ffn_norm", il);
++
++            cur = build_ffn(cur,
++                    layer.ffn_up,   nullptr, layer.ffn_up_s,
++                    layer.ffn_gate, nullptr, layer.ffn_gate_s,
++                    layer.ffn_down, nullptr, layer.ffn_down_s,
++                    nullptr,
++                    LLM_FFN_SILU, LLM_FFN_PAR, il);
++            cb(cur, "ffn_out", il);
++
++            cur = ggml_add(ctx0, cur, ffn_inp);
++            cur = build_cvec(cur, il);
++            cb(cur, "hrm_layer_out", il);
++
++            stack_cur = cur;
++        }
++
++        stack_cur = build_norm(stack_cur, nullptr, nullptr, LLM_NORM_RMS, slot_offset);
++        cb(stack_cur, "stack_final_norm", slot_offset);
++        return stack_cur;
++    };
++
++    for (int h = 0; h < h_cycles; ++h) {
++        for (int l = 0; l < l_cycles; ++l) {
++            const int slot_offset = int((h * (l_cycles + 1) + l) * n_stack);
++            hidden_low = build_stack(ggml_add(ctx0, hidden_low, hidden_high), slot_offset);
++        }
++
++        const int slot_offset = int((h * (l_cycles + 1) + l_cycles) * n_stack);
++        hidden_high = build_stack(ggml_add(ctx0, hidden_high, hidden_low), slot_offset);
++    }
++
++    ggml_tensor * cur = hidden_high;
++
++    if (inp_out_ids) {
++        cur = ggml_get_rows(ctx0, cur, inp_out_ids);
++    }
++
++    res->t_embd = cur;
++
++    cur = build_lora_mm(model.output, cur, model.output_s);
++    cb(cur, "result_output", -1);
++
++    res->t_logits = cur;
++    ggml_build_forward_expand(gf, cur);
++}
+diff --git a/src/models/models.h b/src/models/models.h
+index 7e551eb96..7da6b7f7f 100644
+--- a/src/models/models.h
++++ b/src/models/models.h
+@@ -515,6 +515,20 @@ struct llama_model_qwen3 : public llama_model_base {
+     std::unique_ptr<llm_graph_context> build_arch_graph(const llm_graph_params & params) const override;
+ };
+ 
++struct llama_model_hrm_text : public llama_model_base {
++    llama_model_hrm_text(const struct llama_model_params & params) : llama_model_base(params) {}
++    void load_arch_hparams(llama_model_loader & ml) override;
++    void load_arch_tensors(llama_model_loader & ml) override;
++
++    ggml_tensor * hrm_z_l_init = nullptr;
++
++    struct graph : public llm_graph_context {
++        graph(const llama_model & model, const llm_graph_params & params);
++    };
++
++    std::unique_ptr<llm_graph_context> build_arch_graph(const llm_graph_params & params) const override;
++};
++
+ 
+ struct llama_model_qwen3moe : public llama_model_base {
+     llama_model_qwen3moe(const struct llama_model_params & params) : llama_model_base(params) {}
+diff --git a/tools/server/server-http.h b/tools/server/server-http.h
+--- a/tools/server/server-http.h
++++ b/tools/server/server-http.h
+@@ -2,9 +2,11 @@
+ 
+ #include <atomic>
+ #include <functional>
+ #include <map>
++#include <memory>
+ #include <string>
+ #include <thread>
++#include <unordered_map>
+ #include <vector>
+ #include <cstdint>
+ 

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bc954894ab677dcdc66331863cf12aa4da8aa719fa56538a816632a3935546b
+size 2365606568

special_tokens_map.json

@@ -0,0 +1,6 @@
+{
+  "bos_token": "<|im_start|>",
+  "eos_token": "<|box_end|>",
+  "pad_token": "<|endoftext|>",
+  "unk_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,14 @@
+{
+  "add_prefix_space": null,
+  "backend": "tokenizers",
+  "bos_token": "<|im_start|>",
+  "eos_token": "<|box_end|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 4096,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": "<|endoftext|>",
+  "chat_template": "{{- '<|im_start|><|object_ref_start|>' -}}\n{%- if tools -%}\n{{- '<tools>' -}}\n{{- tools | tojson -}}\n{{- '</tools>\\n' -}}\n{%- endif -%}\n{%- for message in messages -%}\n{%- if message['role'] == 'system' -%}\n{{- '<user>\\n' + message['content'] + '\\n</user>\\n' -}}\n{%- elif message['role'] == 'user' -%}\n{{- '<user>\\n' + message['content'] + '\\n</user>\\n' -}}\n{%- elif message['role'] == 'assistant' -%}\n{{- '<assistant>\\n' -}}\n{%- if message.get('content') -%}\n{{- message['content'] -}}\n{%- endif -%}\n{%- for tool_call in message.get('tool_calls', []) -%}\n{{- '<tool_call>' -}}\n{{- {'arguments': tool_call['function']['arguments'], 'name': tool_call['function']['name']} | tojson -}}\n{{- '</tool_call>' -}}\n{%- endfor -%}\n{{- '\\n' -}}\n{%- elif message['role'] == 'tool' -%}\n{{- '<tool_result>' + message['content'] + '</tool_result>\\n' -}}\n{%- else -%}\n{{- raise_exception('Unsupported message role: ' + message['role']) -}}\n{%- endif -%}\n{%- endfor -%}\n{%- if add_generation_prompt -%}\n{{- '<assistant>\\n' -}}\n{%- endif -%}\n{{- '<|im_end|>' -}}",
+  "clean_up_tokenization_spaces": false
+}