
Gradient Starvation. Why Your LLM Stops Learning
Gradient starvation is a hidden failure mode in machine learning where frequent tokens dominate training, starving rare and informative tokens of updates.
A living index of ways AI can go off the rails. Habsburg AI, Model Autophagy Disorder, Gradient Starvation & more.