metadata
license: apache-2.0
base_model: upstage/Solar-Open-100B
tags:
- pruned
- compressed
- llm
Solar-Open-100B-pruned-5pct
This model is a pruned version of upstage/Solar-Open-100B.
Pruning Details
| Property | Value |
|---|---|
| Original Model | upstage/Solar-Open-100B |
| Original Parameters | 9.46B |
| Pruned Parameters | 3.09B |
| Compression Ratio | 0.3261 (3.1x smaller) |
| Strategy | Layer + Head + FFN (Recommended) |
| Importance Metric | magnitude |
| Layers Removed | 31 |
| FFN Reduction | 60.00% |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")
tokenizer = AutoTokenizer.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")
Notes
This model was created using structured pruning techniques including:
- Layer pruning (removing entire transformer layers)
- Attention head pruning (reducing number of attention heads)
- FFN dimension pruning (reducing intermediate layer sizes)
The pruning was based on magnitude importance scoring to preserve the most important weights.