Instructions to use mradermacher/Ling-Coder-lite-i1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mradermacher/Ling-Coder-lite-i1-GGUF with Transformers:

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mradermacher/Ling-Coder-lite-i1-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mradermacher/Ling-Coder-lite-i1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M

Use Docker

docker model run hf.co/mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use mradermacher/Ling-Coder-lite-i1-GGUF with Ollama:
```
ollama run hf.co/mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use mradermacher/Ling-Coder-lite-i1-GGUF with Docker Model Runner:
```
docker model run hf.co/mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M
```

Lemonade

How to use mradermacher/Ling-Coder-lite-i1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mradermacher/Ling-Coder-lite-i1-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Ling-Coder-lite-i1-GGUF-Q4_K_M

List all available models

lemonade list

Atomic Chat

Request to upload the model weights to ModelScope

by GoZion - opened Jun 6, 2025

Discussion

GoZion

Jun 6, 2025

Hi Mradermacher Team,

We noticed that you have taken the initiative to convert and upload a GGUF-quantized version of Ling-Coder-Lite model weights to Hugging Face — thank you very much for your effort and contribution to the community!

As we're actively expanding the accessibility of our models across different platforms, we would like to ask whether you would be open to also uploading these GGUF-quantized weights to ModelScope (modelscope.cn), which is a widely used model platform in China.

If you are not planning to do so yourself, we would be more than happy to take the responsibility of uploading them on your behalf.

Looking forward to your reply!

Best regards,
Zion
CodeFuse, Ling Team

RichardErkhov

Jun 7, 2025

Hello Zion,

Nice to hear from you. We are actually planning to reupload top 1000 most downloaded models to modelscope in the near future.

I assume the re-upload would start on next week. The profile can be accessed here: https://www.modelscope.cn/profile/mradermacher

I have reuploaded this model and you can access it here: https://www.modelscope.cn/models/mradermacher/Ling-Coder-lite-i1-GGUF

If you have an ideas or suggestions, please let me know.

Best regards,
Richard

GoZion

Jun 9, 2025

Hello Richard,

Thank you for your reply. We are very pleased you have already helped re-upload the weights to Modelscope. Thank you once again!

We greatly appreciate your help and look forward to continuing our collaboration.

Best regards,
Zion

mradermacher

Owner Jun 9, 2025

I'd also like to mention that we don't pretend to have any rights, moral or otherwise, over these quants, so anybody could have uploaded them, of course. But your request has lead to an official representation of mradermacher on modelscope :)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment