Hugging Face's logo
Activity Feed

AI & ML interests

None defined yet.

Recent Activity
osansevieroย  authored a paper 14 days ago
Gemma 4 Technical Report
osansevieroย  submitted a paper 15 days ago
Gemma 4 Technical Report
pdx97ย  authored a paper about 1 month ago
Honest Lying: Understanding Memory Confabulation in Reflexive Agents
View all activity
ajibawa-2023ย 
posted an update 6 days ago
view post
Post
3495
Technical-Architectures-Large
Dataset: ajibawa-2023/Technical-Architectures-Large

This dataset provides over 210,000 distinct enterprise software architectures generated using two open source models: GPT-OSS-120B and Qwen3-Coder-Next-FP8.

These architectures model realistic enterprise systems complete with client layers, edge security, API gateways, service meshes, compliance boundaries, and multi-cloud infrastructure topologies.
osansevieroย 
authored a paper 14 days ago
osansevieroย 
submitted a paper to Daily Papers 15 days ago
mmhamdyย 
posted an update 24 days ago
view post
Post
156
Decades before the modern scaling laws, this paper showed that neural networks behavior under scale follows remarkably predictable laws.

In 1993, researchers at Bell Labs were grappling with a constraint that feels entirely familiar (and contemporary): datasets were outgrowing the available hardware, and training a model to the end was becoming too expensive. To evaluate an architectural tweak to a state-of-the-art model (at the time it was LeNet) on 60,000 samples meant burning up to three weeks of compute time.

To save compute, people would train candidate architectures on small subsets of the data, assuming that the top performer at small scale would remain the top performer at full scale. But with our future wisdom, we know this is not the case.

In "Learning Curves: Asymptotic Values and Rate of Convergence (NeurIPS 93)", using insights from statistical mechanics, they proposed a practical and principled method for predicting the performance of classifiers trained on large datasets (at the time, models were assumed to be large enough). The method was based on a simple power-law modeling of the expected training and test errors.

It is often noted that many of today's breakthroughs in AI and deep learning are actually decades-old concepts that simply lacked the computational power to be tested at the time. While there is some truth to that, it highlights a more valuable lesson: there is immense worth in revisiting early literature and reflecting on foundational ideas we may have prematurely left behind.

So, go explore and find your own inspiration. The current trend has enough champions already!
  • 2 replies
ยท
mmhamdyย 
posted an update 28 days ago
view post
Post
297
It has been more than a decade now since the knowledge distillation paper came out.

Knowledge Distillation (KD) is one of my favorite topics, but I have to confess that I'm not a huge fan of the term because I find it confusing (or at least, it has became so over time).

The idea behind KD is not novel; it was there almost a decade before the paper came out (and arguably even a decade before that, back to 1990-91). But this paper is the one that clicked, the one that made the topic much more popular and introduced it to a broader audience.

First, the timing and the authors played a big role: we have Geoffrey Hinton, Oriol Vinyals, and Jeff Dean here. And second, Geoffrey Hinton is really good at idea branding: Model compression?! No, no, no! Let's call it "Knowledge Distillation" and use evocative terms such as "Dark Knowledge" to describe what is being transferred.

It's a great name, but as time has passed, the term became a bit of a relic. KD is no longer solely about compression (KD used to be introduced as a method for model compression, but now model compression is just one application of KD). And the other thing is that the word "distillation" implies some sort of potency here, that the student is somehow more powerful than the teacher, which is not the case (but many counterarguments could be made, for example, more powerful compared to another model trained with no teacher)

Nevertheless, the paper is incredibly well-written, short, and fun to read. It's one of few papers that I read several times. Check it out, and maybe share your thoughts on the topic with us here!

If you had to choose another name for Knowledge Distillation, what would it be?

  • 6 replies
ยท
ajibawa-2023ย 
posted an update about 1 month ago
view post
Post
6917
Shell-Code-Large
Dataset: ajibawa-2023/Shell-Code-Large

Shell-Code-Large is a large-scale corpus of Shell scripting source code comprising approximately 640,000 code samples stored in JSON Lines (.jsonl) format. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, DevOps automation, cloud infrastructure engineering, system administration, and software engineering automation.

By providing a high-volume, language-specific corpus focused exclusively on Shell scripting, Shell-Code-Large enables systematic experimentation in automation workflows, deployment pipelines, infrastructure management, and command-line tooling. These domains remain foundational to Linux systems, cloud-native platforms, CI/CD environments, and modern DevOps practices.

Shell-Code-Large addresses the need for a dedicated Shell-focused dataset at substantial scale, enabling targeted research into scripting patterns, command composition, workflow orchestration, infrastructure automation, and operational engineering practices
mmhamdyย 
posted an update about 1 month ago
view post
Post
2669
What if you could train a model on just 10 images instead of 60,000 and still get close to the same performance?

Traditional machine learning requires thousands, even millions, of data points to achieve high accuracy. But what if we could "distill" the entire dataset into just a few synthetic samples?

This is what Dataset Distillation offers. Unlike traditional knowledge distillation, we keep the model fixed and distill the knowledge contained in a massive training set into a tiny set of synthetic distilled images.

The goal is to train a model on this ultra-small set and achieve performance that almost matches what the same model would get when trained on the massive original dataset.

For example, training on only 10 distilled MNIST images (this is equivalent to a single image per class) yields 94% accuracy, compared to 99% when training on the full 60,000 images.

Interestingly, these distilled images look significantly different (as you can see in the image below) from natural images because they are optimized for model training rather than for matching the correct data distribution.

But that's not all.

Most importantly, this same method opens the door to a potent form of data poisoning. Because distilled images are specifically optimized for rapid learning, an attacker can create a tiny set of adversarial distilled images to cause a well-trained model to forget or misclassify a specific category.

What I find fascinating about dataset distillation is this: it mimics human-like learning by letting a model grasp a concept from a single example, but it does so using alien synthetic images that mean absolutely nothing to a human eye!

What about you? What are your thoughts on it?
  • 2 replies
ยท
prithivMLmodsย 
posted an update about 1 month ago
view post
Post
7057
Wan2.2-I2V-Fast with highly upscaled sequential frame sampling is now available as a Spaces demo, built using Wan2.2-I2V and FLUX.2-Klein. Try the demo using the links below.๐Ÿ‘‡

โž  wan2.2-i2v-fast : prithivMLmods/wan2.2-i2v-fast
โž  github: https://github.com/prithivsakthiur/wan2.2-i2v-fast
โž  collection: https://huggingface.co/collections/prithivMLmods/image-generation-apps-collection

โคท To learn more, visit the app page or the respective model pages.
pdx97ย 
authored a paper about 1 month ago
mmhamdyย 
posted an update about 1 month ago
view post
Post
4990
It was supposed to be a failed experiment. Instead, it led to the discovery of one of the most intriguing phenomena in neural networks, simply because a researcher forgot to turn it off and left it running....for a week!

In 2022, researchers at OpenAI were studying how neural networks generalize from their training data. For this task, they were training small transformer models to perform modular arithmetic.

The thing is, neural networks are weird. When a model has an abundance of parameters (like neural nets), it can easily overfit. It essentially memorizes its training data, scoring a perfect 100% accuracy when tested on it, but remains completely clueless when faced with any new instances not present in the training set (close to 0 accuracy). It is like memorizing 1 + 2 = 3 without understanding the concept of addition, so if 2 + 3 wasn't in the training set, the model fails miserably!

Usually, when a model overfits like this, people just cut their losses, turn off the experiment, and move on with their lives.

But sometimes they forget. And that is exactly what happened to our researchers at OpenAI. A week later, they checked back in, and a miracle had happened!

They discovered Grokking (And no, this has nothing to do with xAI's Grok , the term was originally coined by sci-fi author Robert Heinlein to mean understanding something so deeply that it becomes part of you). Grokking is when a neural network suddenly and abruptly learns to generalize long after it has overfitted. Just take a look at the graph in the image below!

Spooky, right! I told you neural nets are weird!
  • 4 replies
ยท
pdx97ย 
submitted a paper to Daily Papers about 1 month ago
mmhamdyย 
posted an update about 2 months ago
view post
Post
1030
Human brains don't recreate every pixel to understand the world!

Most current models in genomics, proteomics, and single-cell transcriptomics rely on generative objectives like masked language modeling or next token prediction. While effective, these architectures waste significant capacity reconstructing raw, noisy sequence details that may not carry functional biological meaning.

But a promising, more efficient alternative is emerging: Joint-Embedding Predictive Architecture (JEPA)

Originally introduced by Yann LeCun for computer vision, JEPA is a non-generative, self-supervised learning (SSL) framework. Instead of predicting raw inputs, it operates as a world model that predicts abstract semantic embeddings in latent space.

Recently, the JEPA framework (and its more efficient LeJEPA variant) has been adapted into the biological sciences to develop performing foundation models and to improve on already existing ones.

It's interesting how each adaptation modified and tailored JEPA to suit its specific biological domain, whether by experimenting with different backbones or complementing the objective with other loss terms.

For example, JEPA-DNA and ProteinJEPA used JEPA as a continual pre-training framework to enhance existing foundation models without training from scratch, while Cell-JEPA and JEPA-DNA employed a hybrid objective that combines the JEPA loss with a traditional language modeling loss.

The article below provides an overview of these implementations, along with others that came out this year. As always, your thoughts and feedback are welcome and highly appreciated!

Link to the article is in the first comment ๐Ÿ‘‡
  • 3 replies
ยท
prithivMLmodsย 
posted an update about 2 months ago
mmhamdyย 
posted an update about 2 months ago
view post
Post
142
Things rarely go as we expect!

In 2017, Google released the Transformer architecture. While it was clear the model was promising, absolutely no one (including its authors) anticipated the pervasive global revolution it would create!

The authors actually viewed the Transformer as just a stepping stone for a much more ambitious project: The MultiModel.

Their ultimate goal was to build a single deep learning architecture capable of jointly learning massive, diverse tasks across entirely different domains (in 2017). A One Model To Learn Them All.

In fact, the MultiModel paper was published in the exact same month as Attention Is All You Need!

But history had other plans. The building block eclipsed the grand design!

So, have you heard about the MultiModel before? ๐Ÿ˜€
  • 1 reply
ยท
alielfilali01ย 
posted an update about 2 months ago
view post
Post
642
Plans in HTML > Plans in Markdown
prithivMLmodsย 
posted an update about 2 months ago
view post
Post
6260
PiD โ€” Pixel Diffusion Decoder Image Edit Upscale and Image Generation Upscale, an all-in-one demo, is now live on Spaces! Great improvements in realism-based image generation and editing are powered by FLUX.2-Klein, while image generation is paired with Z-Image, and upscaling is enabled by default!

๐Ÿค— Space: prithivMLmods/PiD-Image-Upscaler
๐Ÿ”— Collection: https://huggingface.co/collections/prithivMLmods/image-generation-apps-collection

๐Ÿค— > To learn more, visit the app page or the respective model pages.
Locutusqueย 
posted an update about 2 months ago
view post
Post
386
๐Ÿš€ Introducing Esmeralda-Llama-3.1-8B-control
The first release in the Esmeralda model family by Locutusque.

This model is intentionally small and experimental โ€” a control/baseline proof-of-concept designed to answer one question:

ยซโ€œHow strong is my new "Locutusque/esmeralda-agentic" dataset before scaling to larger runs?โ€ยป

Training Details

- Base: Llama 3.1 8B
- Training precision: bf16 mixed precision
- Chat template: modified ChatML
- Dataset size: ~37k examples
- Examples actually used for this run: ~5k

The dataset includes:

- multi-turn agentic traces
- reasoning traces
- structured assistant behavior
- generalist instruction data

Benchmark Results

Compared against:

- Llama 3.1 8B Instruct
- Hermes-3-Llama-3.1-8B

HumanEval

57.3 โ€” Esmeralda
56.1 โ€” Llama 3.1 Instruct
52.4 โ€” Hermes-3

MBPP

53.2 โ€” Esmeralda
56.8 โ€” Llama 3.1 Instruct
48.2 โ€” Hermes-3

GPQA Diamond

15.7 โ€” Esmeralda
15.7 โ€” Llama 3.1 Instruct
18.2 โ€” Hermes-3

EQ-Bench

59.2 โ€” Esmeralda
61.1 โ€” Llama 3.1 Instruct
63.1 โ€” Hermes-3

EQ-Bench Parseable (Syntax Stability)

๐Ÿ”ฅ 100.0% โ€” Esmeralda
92.4% โ€” Llama 3.1 Instruct
91.2% โ€” Hermes-3

Here Be Dragons ๐Ÿ‰

I also experimented with a new TruthfulQA free-generation evaluation setup.

- Responses were judged by Gemma 4 26B A4B
- The judge compared generations directly against ground-truth answers
- Models were evaluated in 8-bit quantized form to speed up inference

TruthfulQA (LLM Judge)

0.682 โ€” Esmeralda-Llama-3.1-8B-control
0.587 โ€” Hermes-3-Llama-3.1-8B (reported MC2 score; methodology differs)

For a lightweight control run trained on only a fraction of the dataset, Iโ€™m pretty encouraged by the results.

The model is released under the standard Llama 3.1 license, and Iโ€™d genuinely love feedback from people testing it in real workflows.

Model: Locutusque/Esmeralda-Llama-3.1-8B-control

Dataset: Locutusque/esmeralda-agentic
alvarobarttย 
posted an update 2 months ago
view post
Post
487
Open agents on AWS SageMaker AI with open models from the Hugging Face Hub!

> Deploy an open model from the Hugging Face Hub on SageMaker AI
> Connect the deployed model to Strands Agents
> Add built-in and custom tools for tool calling
> Expose external capabilities through MCP integration
> Bonus: talk to your agent and visualize traces with Gradio

https://alvarobartt.com/agents-on-aws-sagemaker
prithivMLmodsย 
posted an update 2 months ago
view post
Post
5616
I've made 8 Spaces in the Qwen-Image-Edit series, and out of them, 5 Spaces reached โ€œSpace of the Weekโ€! A few Spaces are still topping the list even after many months.

Cumulatively, the series has crossed 8.2 million+ ZeroGPU runs and nearly 4 million visitors overall.

Thanks for all the community support! ๐Ÿค—โค๏ธ

๐Ÿ”— Spaces: https://huggingface.co/collections/prithivMLmods/image-generation-apps-collection
  • 4 replies
ยท
alvarobarttย 
posted an update 2 months ago
view post
Post
3357
Latest hf-mem release added a breakdown of Mixture-of-Experts (MoE) memory usage!

TL; DR MoEs can be misleading to reason about from active parameters alone, since each token only activates a subset of experts, while the serving setup still needs to account for the full resident memory footprint.

๐Ÿง  hf-mem now splits MoE memory into base model weights, routed experts, and KV cache
๐Ÿ—๏ธ Dense models usually load and use most weights every forward pass, while MoEs load many experts but only route each token to a few of them
โšก Active params isn't the same as memory footprint, especially for sparse architectures
๐Ÿ“ฆ Runtime memory is about what is used per request/token, while loading memory also includes the expert weights that need to be resident
๐Ÿ“š KV cache can still dominate depending on context length, batch size, and concurrency
๐Ÿ”€ Expert Parallelism (EP) helps shard experts across accelerators when expert weights dominate
๐Ÿš€ Data Parallelism (DP) + EP is often a good fit for throughput-oriented MoE serving

Check the repository at https://github.com/alvarobartt/hf-mem