Nanbeige4.2-3B-oQ4e
Apple Silicon Optimized oQ4e MLX Quantized Release
This repository contains an oQ4e mixed-precision MLX quantized version of Nanbeige4.2-3B, optimized for fast and memory-efficient local inference on Apple Silicon.
The original Nanbeige4.2-3B model was developed by the Nanbeige LLM Lab. This repository contains an optimized MLX/oQ4e conversion only and does not include any additional fine-tuning or retraining.
Using oQ4e sensitivity-aware mixed-precision quantization, this release preserves the strong agentic reasoning, coding, and tool-use capabilities of the original model while significantly reducing memory usage for local deployment.
About Nanbeige4.2
Nanbeige4.2-3B is a compact agentic language model built upon Nanbeige4.2-3B-Base.
Despite having only 3B non-embedding parameters, it demonstrates remarkably strong performance across agentic reasoning, coding, office automation, and tool-use tasks.
A major innovation of Nanbeige4.2 is its Looped Transformer architecture, where hidden representations are passed back through the transformer stack after a complete forward pass. This architectural reuse increases effective model capacity without increasing parameter count.
The original model combines:
- Large-scale supervised fine-tuning
- Real-world environment interaction
- Synthetic environment generation
- Outcome-based reinforcement learning
- Process reward optimization
- Tool-use supervision
- Code-agent training
- Office-agent workflows
The result is a compact model capable of competing with significantly larger language models on many practical agent benchmarks.
Quantization
This release uses oQ4e mixed-precision quantization.
Specifications
- Format: MLX
- Quantization: oQ4e
- Method: Sensitivity-Aware Mixed Precision
- Target Platform: Apple Silicon
- Inference Engine: MLX / oMLX
Unlike conventional fixed-bit quantization, oQ4e dynamically allocates precision according to layer sensitivity, preserving critical reasoning and planning layers while aggressively compressing less sensitive portions of the network.
Benefits include:
- Excellent reasoning preservation
- Strong coding performance
- Efficient tool use
- Reduced memory usage
- Faster inference
- Optimized Apple Silicon execution
Example Usage
from mlx_lm import load, generate
model, tokenizer = load("yugeshkarunamurthy/Nanbeige4.2-3B-oQ4e")
messages = [
{
"role": "user",
"content": "Design a task planner for a coding assistant."
}
]
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
enable_thinking=True,
)
response = generate(
model,
tokenizer,
prompt=prompt,
temp=0.7,
top_p=0.95,
top_k=20,
max_tokens=16384,
)
print(response)
Optimized For
This release is optimized for:
- Apple M1
- Apple M2
- Apple M3
- Apple M4
Compatible with:
- MLX
- MLX-LM
- oMLX
- Open WebUI
- LM Studio (MLX)
- Local AI Agent Frameworks
Model Highlights
- Looped Transformer Architecture
- Strong Agentic Reasoning
- Native Tool Use
- Function Calling
- Software Engineering
- Office Automation
- Multi-step Planning
- Coding Assistance
- Research Assistance
- Instruction Following
Intended Use
Nanbeige4.2-3B-oQ4e is well suited for:
- AI Agents
- Coding Assistants
- Tool Calling
- Software Development
- Research Automation
- Multi-step Planning
- Office Productivity
- Workflow Automation
- Local AI Applications
- General Purpose Reasoning
Hardware Recommendations
Recommended systems:
- Apple M1 Pro / Max / Ultra
- Apple M2 Pro / Max / Ultra
- Apple M3 Series
- Apple M4 Series
The oQ4e quantization significantly reduces memory usage, making the model practical for a wide range of Apple Silicon devices while maintaining excellent reasoning quality.
About oQ4e Quantization
oQ4e is a sensitivity-aware mixed-precision quantization technique designed to preserve model quality while substantially reducing memory requirements.
Rather than assigning identical precision to every layer, oQ4e analyzes layer sensitivity and selectively preserves higher precision where it has the greatest impact on reasoning and generation quality.
Benefits include:
- Better reasoning retention
- Stronger planning capabilities
- Higher coding accuracy
- Lower RAM usage
- Faster inference
- Excellent Apple Silicon optimization
Original Model
Nanbeige4.2-3B introduces a Looped Transformer architecture, where hidden states are reused through the transformer layers to increase effective model capacity without increasing parameter count. The model is trained with a combination of supervised fine-tuning and reinforcement learning to strengthen agentic behavior, tool use, and coding performance. It is released under the Apache-2.0 license. :contentReference[oaicite:0]{index=0}
Credits
Original Model
All credit for the original model, datasets, training methodology, evaluation, benchmarks, and research belongs entirely to:
Nanbeige LLM Lab
Original Repository:
https://huggingface.co/Nanbeige/Nanbeige4.2-3B
Base Model:
https://huggingface.co/Nanbeige/Nanbeige4.2-3B-Base
oQ4e MLX Quantized Release
This repository provides an Apple Silicon optimized oQ4e MLX quantized version of the original model.
No additional fine-tuning has been performed.
Acknowledgements
- Nanbeige LLM Lab
- Apple MLX
- Hugging Face
- Transformers
- vLLM
- SGLang
- MLX-LM
- oMLX
- OptiQ Quantization
Citation
If you use this model in research, please cite the original Nanbeige4.2 model:
@misc{nanbeige2026,
title = {Nanbeige4.2-3B},
author = {Nanbeige Team},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/Nanbeige/Nanbeige4.2-3B}}
}
License
This release inherits the Apache-2.0 license from the original model.
Please refer to the original repository for complete licensing information.
Disclaimer
This repository contains an optimized oQ4e MLX quantized conversion intended for efficient local inference on Apple Silicon.
All original model architecture, datasets, training methodology, benchmarks, evaluations, and research remain entirely the work of the original Nanbeige LLM Lab.
- Downloads last month
- 91
4-bit