Qwen3.8-4B-A1-DARE-TIES

Configuracao do Merge

Parametro Valor
Metodo DARE-TIES
Base Qwen/Qwen3.5-4B
Fonte 1 empero-ai/Qwen3.8-4B-Distill (weight=0.7)
Fonte 2 InternScience/Agents-A1-4B (weight=0.3)
Density 0.7
Seed 42
Shards 2 (5.01 GB + 3.67 GB)
Total 8.68 GB (bf16)

Objetivo do Merge

Combinar a deducao logica do Qwen3.8-4B-Distill (78% em BBH Logical Deduction) com a capacidade agentica do Agents-A1-4B (38% em BBH Causal Judgement).

Benchmarks dos Modelos Pais

Modelo Deducao Causal Medicina
Qwen3.8-4B-Distill 78% 46% 88%
Agents-A1-4B 26% 38% 88%
Este merge (0.7/0.3) a medir a medir a medir

Uso

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
)

model = AutoModelForCausalLM.from_pretrained(
    "leandronunes/qwen3.8-4b-a1-dare-ties",
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True,
    attn_implementation="sdpa",
)
tokenizer = AutoTokenizer.from_pretrained(
    "leandronunes/qwen3.8-4b-a1-dare-ties",
    trust_remote_code=True,
)

Notas Tecnicas

  • Modelo multimodal (VLM): inclui torre de visao de 24 blocos alem do language model.
  • Arquitetura hibrida: 3 camadas linear_attention (Gated DeltaNet) seguidas de 1 full_attention, repetido 8 vezes.
  • Thinking mode: para desativar o raciocinio estendido, use enable_thinking=False no apply_chat_template.
  • Contexto: 262.144 tokens (256K).

Licenca

Apache 2.0 (herdada do Qwen3.5-4B).

Downloads last month
18
Safetensors
Model size
5B params
Tensor type
BF16
ยท
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for leandronunes/qwen3.8-4b-a1-dare-ties