🤔 On to something

Daniel Fox PRO

FlameF0X

121 192 402

https://flamef0x.github.io

FlameF0X

AI & ML interests

Pre-training text generator. (Brother, im 18) Please don't try to contact me.

Recent Activity

reacted to sergiopaniego's post with 🔥 13 minutes ago

you can train DiffusionGemma (a block-diffusion LLM) in TRL! and we're sharing an example for it TRL trainers are made to be easily extended and adapted to different real-world use cases. in this one, with a single method overridden in SFTTrainer (compute_loss), you can train this model > example: https://github.com/huggingface/trl/blob/main/examples/scripts/sft_diffusion_gemma.py

new activity about 3 hours ago

wop/FWKV-ROSA:200tokens/s

liked a Space about 3 hours ago

wop/FWKV-ROSA

View all activity

Organizations

Hello, people of Hugging Face!

I recently released FlameF0X/TinyMoE-100m-2x8-retrained, a small Mixture of Experts language model trained on the Smollm-Corpus. Built on top of the Mixtral architecture, it’s fully compatible with 🤗 Transformers right out of the box!

The model can produce somewhat coherent text on its own, and for some reason, it generates even more coherent responses when given a ChatLM template.

I’m excited to see what you all come up with, and feel free to fine-tune it if you’d like. In the meantime, I’ll be working on developing the chat-trained version.

Demo: FlameF0X/TinyMoE-Playground
Collection: https://huggingface.co/collections/FlameF0X/tinymoe

Post

353

My models on the Intel Low-Bit LLM Leaderboard

Figured I'd share where my quantized models landed on Intel/low_bit_open_llm_leaderboard since I hadn't posted about it yet.

FlameF0X/Qwen3-4B-Distilled-Claude-4.6 (NVFP4 and MXFP4) sit at ranks 23 and 24 with 62.68% and 61.18% average, right below the base Qwen3-4B. Not bad considering they were distilled from Claude 4.6 rather than trained from scratch.

FlameF0X/LFM2.5-1.2B-Distilled-Claude-4.6 and FlameF0X/LFM2.5-1.2B-Thinking-CodeX land around rank 47-49, competitive with MiniCPM5-1B and the Qwen3 sub-1B models despite being a larger base architecture.

The funny one is FlameF0X/Qwen2-0.2B-pt and FlameF0X/Qwen2-0.2B-it. They're not properly trained — genuinely undertrained, basically undefined — and they still beat openai/gpt-oss-20b at rank 66. The 20B model. Not sure what that says but it's something.

FlameF0X/LFM2-Research is at the bottom of my lineup but it's a research artifact, not meant to be competitive.

Chart below showing my models vs nearby competitors, with size vs performance on the left.

Chart made by Claude