trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-4Bit

Qwimi3.5-9B KimiK2.6 Opus Distill MLX 4-bit

This is the MLX 4-bit conversion of:

trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16

It is intended for Apple Silicon / MLX users who want to run the Qwimi3.5-9B reasoning fine-tune locally.

This repo is an MLX-format conversion/quantization of the merged BF16 model. The training was done separately using Unsloth + LoRA SFT, then merged to BF16 before MLX conversion.

---

Model Summary

Field	Value
Model name	Qwimi3.5-9B-Kimik2.6-Opus-Distill
Format	MLX 4-bit
Source model	trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16
Base model	unsloth/Qwen3.5-9B
Training method	LoRA SFT
LoRA rank	r = 128
LoRA alpha	128
RS-LoRA	Enabled
Training context length	16k
Train examples	12,000
Held-out eval examples	366
Trainer eval subset	200
Epochs	1
Final train loss	0.5517
Lightweight eval loss	~0.3161

---

Quantization Notes

This repository contains the 4-bit MLX version.

The 4-bit version is usually the best first choice for most Apple Silicon users because it offers the smallest size and lowest memory footprint. It is the most practical option for users with tighter RAM limits.

For other formats, see the main release family:

• BF16: trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16
• GGUF: trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-GGUF

---

Install

pip install -U mlx-lm

---

Basic Usage

mlx_lm.generate \
  --model trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-4bit \
  --prompt "Explain RS-LoRA in simple terms."

For longer generations:

mlx_lm.generate \
  --model trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-4bit \
  --prompt "Solve this step by step: If a company grows revenue from 120M to 180M over 3 years, what is the CAGR?" \
  --max-tokens 1024

---

Chat Template / Prompting

This model was trained with Qwen-style chat formatting and response-only SFT. For best results, use an instruction/chat style prompt.

Example:

You are a helpful reasoning assistant.

Question:
Explain why RS-LoRA can help at higher LoRA ranks.

The model was trained on reasoning-style outputs where assistant responses often contain:

<think>
reasoning trace
</think>
final answer

Depending on your inference settings and prompt, the model may produce explicit reasoning traces. For end-user applications, you may want to post-process or hide internal reasoning and show only the final answer.

---

Training Data

The merged BF16 source model was trained on a combined reasoning/distillation mix from:

1. trjxter/Kimi-K2.6-Reasoning-3300x-WandB
2. Jackrong/Qwen3.5-reasoning-700x
3. Jackrong/Claude-opus-4.6-TraceInversion-9000x

After formatting and 16k-token filtering, the final usable dataset size was:

12,366 usable examples
12,000 train examples
366 held-out eval examples
200 trainer eval examples

The curation process normalized mixed schemas into a shared Qwen chat-template SFT format.

---

Training Configuration

The original fine-tune used:

Base model: unsloth/Qwen3.5-9B
Training: LoRA SFT
Base loading: 8-bit
LoRA rank: 128
LoRA alpha: 128
RS-LoRA: enabled
Max sequence length: 16,384
Gradient checkpointing: unsloth
Batch size: 2
Gradient accumulation: 8
Effective batch size: 16
Epochs: 1
Optimizer: adamw_8bit
Learning rate: 2e-5
Warmup steps: 30
Save steps: 200
Eval steps: 200

Expected optimizer steps:

12,000 / 16 = 750 optimizer steps

---

Training Results

Final trainer metrics from the source fine-tune:

{
    "train_runtime": 15728.7981,
    "train_samples_per_second": 0.763,
    "train_steps_per_second": 0.048,
    "total_flos": 1.4528477736160543e+18,
    "train_loss": 0.5516796213785807,
    "epoch": 1.0,
}

Loss checkpoints:

Step	Train Loss	Lightweight Eval Loss
200	0.4149	0.3284
400	0.5867	0.3216
600	0.5480	0.3177
750	0.5531	0.3161

---

Artifact Lineage

flowchart TD
    A[unsloth/Qwen3.5-9B] --> B[LoRA SFT]
    B --> C[Qwimi LoRA adapter]
    C --> D[Merged BF16 model]
    D --> E[MLX 4-bit conversion]

---

Intended Use

This model is intended for:

• general reasoning experiments,
• local Apple Silicon inference,
• MLX benchmarking,
• educational fine-tuning research,
• testing Qwen-style reasoning distillation behavior.

It is not intended for high-stakes medical, legal, financial, or safety-critical decision-making without expert review.

---

Limitations

• This is a community fine-tune and should be evaluated before production use.
• The model may produce incorrect reasoning or confident wrong answers.
• The model may generate verbose reasoning traces depending on prompts.
• MLX quantized behavior can differ from BF16 or GGUF behavior.
• This README reports the training metrics of the BF16 source model, not a separate full eval of this exact MLX quant.

---

Citation / Attribution

Base model and tooling credit goes to the Qwen, Unsloth, Hugging Face, and MLX communities.

Source fine-tune:

trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16

MLX conversion:

trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-MLX-4bit

---

Notes

This MLX quant was created to make the Qwimi3.5-9B run easier to use for Apple Silicon users.