How to use from
SGLang
Install from pip and serve model
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "budecosystem/boomer-634m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "budecosystem/boomer-634m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'
Use Docker images
docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "budecosystem/boomer-634m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "budecosystem/boomer-634m",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'
Quick Links

Democratizing access to LLMs for the open-source community.
Let's advance AI, together.


Introduction πŸŽ‰

We are thrilled to announce the open-sourcing of our boomer-634m model, an important milestone in our ongoing AI research. This model, with 634 million parameters, was meticulously pre-trained from scratch on a custom synthetic dataset comprising 12 billion tokens.

Run the model

Here is a quick guide to get you started with boomer-634m: Please note that, at the moment, trust_remote_code=True is required for running the model.

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("budecosystem/boomer-634m",
                                             trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("budecosystem/boomer-634m")

input_ids = tokenizer("Explain why the sky is blue.", return_tensors='pt').to(model.device)["input_ids"]
outputs = model.generate(input_ids, max_new_tokens=216)
print(tokenizer.batch_decode(outputs))

Evaluations

The boomer-634m model has been rigorously evaluated on various benchmarks, showcasing its robust performance across different tasks:

Model Name MMLU ARC Hellaswag GSM8K Winogrande MathQA logiqa
boomer-634m 25.91 29.86 39.24 1.67 50.67 23.55 28.42

Final thought on Boomer!

Embarking on the journey with boomer-634m is just the beginning. We are committed to developing more advanced, efficient, and accessible AI models. Join us in this exciting adventure to shape the future of AI.

Aknowledgements

Our heartfelt thanks go to the open-source community and the trailblazers in AI research whose work has paved the way for innovations like boomer-634m.

Downloads last month
292
Safetensors
Model size
0.6B params
Tensor type
F16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Dataset used to train budecosystem/boomer-634m