Devstral Small 2 docs proposal: local Mac API with Mistral tool-format boundaries

#33
by Raullen - opened

Proposing a community API-serving example alongside the Devstral Small 2 local deployment options. The client would connect to Rapid-MLX on loopback using /v1/models and /v1/chat/completions.

Rapid's model routing distinguishes native Mistral/Devstral tool syntax from Hermes-format fine-tunes on a Mistral base. A recipe should preserve that distinction rather than copy a generic Hermes parser flag. It should also use the model card's sampling guidance and test a complete tool call plus the tool-result response.

The proposed initial scope is text and tool-message transport. Screenshot understanding would require its own image preprocessing and output checks; it should not be inferred from a successful chat request. This is a request to agree on a useful community example, not a claim of SWE-bench reproduction.

Concrete catalog entry: devstral-v2-24b-4bit → mlx-community/Devstral-Small-2-24B-Instruct-2512-4bit@97228558fcbdd1adefd39d515155db3a894b37f7. This is a conversion/distribution artifact; it is not presented as an official upstream release.

Source references (fixed Rapid commit): catalog, family routing, native parser.

Verification status: this proposal is based on reading the catalog, implementation and the linked documentation/tests. No fresh model inference or test-suite run was performed for this submission; source tests are not an end-to-end qualification. It requests a community documentation scope, not an official endorsement or a performance listing.

Disclosure: I maintain Rapid-MLX. Prepared with AI assistance under my direction. Anonymous usage-metadata telemetry is enabled by default; DO_NOT_TRACK=1 disables it. The original model and conversion terms continue to apply.

Sign up or log in to comment