https://mosthumble.github.io/Sifal KliouiA blog about Machine learning, programming, and other interesting things. 2026-04-24T22:27:46+02:00 sifal https://mosthumble.github.io/ Jekyll © 2026 sifal /assets/img/favicons/favicon.ico /assets/img/favicons/favicon-96x96.png Towards a Bitter Lesson of Optimization: When Neural Networks Write Their Own Update Rules2026-04-07T16:47:50+02:00 2026-04-24T22:25:45+02:00 https://mosthumble.github.io/posts/Towards-a-Bitter-Lesson-of-Optimization-When-Neural-Networks-Write-Their-Own-Update-Rules/ sifal We explore what can be the future of neural network parameter optimization Why Modern LLMs Dropped Mean Centering (And Got Away With It)2026-02-22T02:00:00+01:00 2026-02-23T09:52:20+01:00 https://mosthumble.github.io/posts/Why-Modern-LLMs-Dropped-Mean-Centering-(And-Got-Away-With-It)/ sifal Visualizing the hidden 3D geometry behind Layer Normalization and uncovering the mathematical trick that makes RMSNorm tick. The Epsilon Trap: When Adam Stops Being Adam2026-01-17T07:30:00+01:00 2026-02-17T20:56:01+01:00 https://mosthumble.github.io/posts/The-Epsilon-Trap-When-Adam-Stops-Being-Adam/ sifal Beyond numerical stability, we investigate an often overlooked hyperparameter in the Adam optimizer: epsilon. Entropic Instruction Following: Does Semantic Coherence Help LLMs Follow Instructions?2025-12-02T09:01:00+01:00 2025-12-07T18:41:42+01:00 https://mosthumble.github.io/posts/Entropic-Instruction-Following/ sifal Testing whether semantic relatedness of instructions affects a model's ability to follow them under cognitive load Elements Of Mechanistic Interpretability: From Observation to Causation2025-10-26T16:45:00+01:00 2025-10-29T22:59:39+01:00 https://mosthumble.github.io/posts/Elements-Of-Mechanistic-Interpretability-From-Observation-to-Causation/ sifal We strip down mechanistic interpretability to three key experiments: watching a model 'think', finding where it stores concepts, and performing 'causal surgery' to change its 'thought process'