spark-darwin-6way

Darwin uniform-soup merge (convex per-layer-block combination) of 6 Spark2.5-4B donors, all verified 290/290 tensors, identical shapes:

  • sriq-ai/sriq-spark-v1.4 (reasoning traces)
  • hcnote/Spark-X2.5-4B-Writing-EP1 (writing EP1)
  • hcnote/SparkMuse-4B (writing final)
  • soyaakinohara/Spark-X2.5-4B-Heretic (refusal-attenuated)
  • Takenoko12345678/Spark-X2.5-4B-Japanese (Japanese)
  • OceanLabs/Ocean-1-4B (general/agentic)

Method: theta_M = sum_i w_i theta_i, uniform w, 38 layer-block groups. Evo search over the genome comes next; this is the generation-0 baseline.

Verified specs: 36 layers, hidden 2560, intermediate 10240, 16 heads / 4 KV, vocab 131072, 290 tensors, ~4.11B. Full program plan: https://github.com/shadyuwugurl/spark-pipeline (SPARK_PLAN.md).

Needs transformers==5.17.0 + trust_remote_code=True. Custom code note: modeling_spark.py carries two compat shims vs stock (mask kwarg rename, dropped cache_position); config.json rope_parameters rewritten validator-clean with identical theta/factor values.

Eval battery in progress (hellaswag/arc_challenge/mmlu x200); numbers land here.

Downloads last month
433
Safetensors
Model size
4B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for MC7ever/spark-darwin-6way