dwarezBrowse by topic
Topics
Posts grouped by the tags already attached to each article.
ai9
- Transformer Attention, Backwards
/ You already know what a language model does. Let's reverse-engineer how.
- Pipeline Parallelism: Surgery on Models Too Big for the Operating Table
/ In our latest article, we explored Data Parallelism.
- Data Parallelism: Scaling LLM Training Through Parallel Processing
/ In my latest article , I discussed the theoretical memory usage needed for inference and training with LLMs, highlighting the memory cost of each component involved in the process.
- Move Fast or Die Slow
/ Today’s article steps back from our usual technical deep-dives to examine the strategic importance of ML optimization.
- The Operating Room Setup
/ Undoubtedly, one of the most critical aspects of machine learning is understanding the theory—without grasping how machines learn, you’ll never excel as an ML Surgeon!
- A quick incision: ten minutes to RAG
/ In under 10 minutes, you’ll discover what RAG is, how to build a prototype for it in Python, and—most importantly—the true weight of the infamous Mr. Fat Raccoon.
- Performing Kernel Surgery: Profiling CUDA Kernels with NVIDIA Nsight Compute
/ Being a Machine Learning Surgeon is not an easy life.
- A Machine Learning Surgeon’s Toolkit: Advanced Matrix Multiplication in CUDA
/ If you want to learn how to write a CUDA kernel for matrix multiplication, look no further!
- An Introduction to Sparsity for Efficient Neural Network Inference
/ Sparsity is a solution to reduce the number of parameters and number of operations in Neural Networks, granting outstanding computational speedups and memory savings during inference.
attention1
- What Kimi K3 asks of the hardware
/ Reading the Kimi K3 announcement from the systems side. How linear attention changes the KV-cache budget, what 896 experts mean for the interconnect, and why the model ships in MXFP4.
compilers1
- Dissecting torch.compile: Surgical Precision in PyTorch Optimization
/ You can take a look at the GitHub repository of this blogpost at this link
cuda6
- A fused Lion optimizer kernel with Hugging Face kernels
/ I wrote a fused Lion optimizer step with Hugging Face kernels, packaged it with kernel-builder, and tested it on CUDA and Metal/MPS.
- The Operating Room Setup
/ Undoubtedly, one of the most critical aspects of machine learning is understanding the theory—without grasping how machines learn, you’ll never excel as an ML Surgeon!
- Performing Kernel Surgery: Profiling CUDA Kernels with NVIDIA Nsight Compute
/ Being a Machine Learning Surgeon is not an easy life.
- A Machine Learning Surgeon’s Toolkit: Advanced Matrix Multiplication in CUDA
/ If you want to learn how to write a CUDA kernel for matrix multiplication, look no further!
- Cerebral Cortex and Hippocampus: Understanding the Computational and Memory Design of GPUs
/ Just as a surgeon needs to understand anatomy, knowing GPU architecture is key to writing efficient kernels. You don’t need to be a hardware expert—just grasp the basics to unlock their full potential
- Hello CUDA: A Surgical Dissection
/ CUDA enables developers to harness NVIDIA GPUs for general-purpose tasks. This article guides you to a "Hello, World!" program as a starting point.
efficiency11
- GGUF quantization, bit by bit
/ Q8_0 to IQ1_S and everything in between. Block layouts, scale hierarchies, codebooks, the importance matrix, and the mixed recipes used by llama.cpp, Unsloth, and DwarfStar.
- TurboQuant: What 3-Bit KV Caches Actually Mean for Your Inference Stack
/ A quick look to Google's new quantization method, with a bit of drama
- Pipeline Parallelism: Surgery on Models Too Big for the Operating Table
/ In our latest article, we explored Data Parallelism.
- Data Parallelism: Scaling LLM Training Through Parallel Processing
/ In my latest article , I discussed the theoretical memory usage needed for inference and training with LLMs, highlighting the memory cost of each component involved in the process.
- The Operating Room Setup
/ Undoubtedly, one of the most critical aspects of machine learning is understanding the theory—without grasping how machines learn, you’ll never excel as an ML Surgeon!
- A quick incision: ten minutes to RAG
/ In under 10 minutes, you’ll discover what RAG is, how to build a prototype for it in Python, and—most importantly—the true weight of the infamous Mr. Fat Raccoon.
- Performing Kernel Surgery: Profiling CUDA Kernels with NVIDIA Nsight Compute
/ Being a Machine Learning Surgeon is not an easy life.
- A Machine Learning Surgeon’s Toolkit: Advanced Matrix Multiplication in CUDA
/ If you want to learn how to write a CUDA kernel for matrix multiplication, look no further!
- Cerebral Cortex and Hippocampus: Understanding the Computational and Memory Design of GPUs
/ Just as a surgeon needs to understand anatomy, knowing GPU architecture is key to writing efficient kernels. You don’t need to be a hardware expert—just grasp the basics to unlock their full potential
- Hello CUDA: A Surgical Dissection
/ CUDA enables developers to harness NVIDIA GPUs for general-purpose tasks. This article guides you to a "Hello, World!" program as a starting point.
- An Introduction to Sparsity for Efficient Neural Network Inference
/ Sparsity is a solution to reduce the number of parameters and number of operations in Neural Networks, granting outstanding computational speedups and memory savings during inference.
gguf1
- GGUF quantization, bit by bit
/ Q8_0 to IQ1_S and everything in between. Block layouts, scale hierarchies, codebooks, the importance matrix, and the mixed recipes used by llama.cpp, Unsloth, and DwarfStar.
gpu1
- From Sequential to Parallel: Your Journey into GPU Programming with Triton
/ We all know that GPU programming is hard.
grpo1
- GRPO is mostly a systems problem
/ The GRPO config is small. The hard part is keeping rollout workers, trainers, rewards, and policy versions in the same story.
hardware1
- What Kimi K3 asks of the hardware
/ Reading the Kimi K3 announcement from the systems side. How linear attention changes the KV-cache budget, what 896 experts mean for the interconnect, and why the model ships in MXFP4.
inference7
- What Kimi K3 asks of the hardware
/ Reading the Kimi K3 announcement from the systems side. How linear attention changes the KV-cache budget, what 896 experts mean for the interconnect, and why the model ships in MXFP4.
- GGUF quantization, bit by bit
/ Q8_0 to IQ1_S and everything in between. Block layouts, scale hierarchies, codebooks, the importance matrix, and the mixed recipes used by llama.cpp, Unsloth, and DwarfStar.
- TurboQuant: What 3-Bit KV Caches Actually Mean for Your Inference Stack
/ A quick look to Google's new quantization method, with a bit of drama
- Transformer Attention, Backwards
/ You already know what a language model does. Let's reverse-engineer how.
- Pipeline Parallelism: Surgery on Models Too Big for the Operating Table
/ In our latest article, we explored Data Parallelism.
- A quick incision: ten minutes to RAG
/ In under 10 minutes, you’ll discover what RAG is, how to build a prototype for it in Python, and—most importantly—the true weight of the infamous Mr. Fat Raccoon.
- An Introduction to Sparsity for Efficient Neural Network Inference
/ Sparsity is a solution to reduce the number of parameters and number of operations in Neural Networks, granting outstanding computational speedups and memory savings during inference.
kernels1
- A fused Lion optimizer kernel with Hugging Face kernels
/ I wrote a fused Lion optimizer step with Hugging Face kernels, packaged it with kernel-builder, and tested it on CUDA and Metal/MPS.
metal1
- A fused Lion optimizer kernel with Hugging Face kernels
/ I wrote a fused Lion optimizer step with Hugging Face kernels, packaged it with kernel-builder, and tested it on CUDA and Metal/MPS.
ml20
- What Kimi K3 asks of the hardware
/ Reading the Kimi K3 announcement from the systems side. How linear attention changes the KV-cache budget, what 896 experts mean for the interconnect, and why the model ships in MXFP4.
- GGUF quantization, bit by bit
/ Q8_0 to IQ1_S and everything in between. Block layouts, scale hierarchies, codebooks, the importance matrix, and the mixed recipes used by llama.cpp, Unsloth, and DwarfStar.
- GRPO is mostly a systems problem
/ The GRPO config is small. The hard part is keeping rollout workers, trainers, rewards, and policy versions in the same story.
- A fused Lion optimizer kernel with Hugging Face kernels
/ I wrote a fused Lion optimizer step with Hugging Face kernels, packaged it with kernel-builder, and tested it on CUDA and Metal/MPS.
- TurboQuant: What 3-Bit KV Caches Actually Mean for Your Inference Stack
/ A quick look to Google's new quantization method, with a bit of drama
- Transformer Attention, Backwards
/ You already know what a language model does. Let's reverse-engineer how.
- Pipeline Parallelism: Surgery on Models Too Big for the Operating Table
/ In our latest article, we explored Data Parallelism.
- Data Parallelism: Scaling LLM Training Through Parallel Processing
/ In my latest article , I discussed the theoretical memory usage needed for inference and training with LLMs, highlighting the memory cost of each component involved in the process.
- The Memory Anatomy of Large Language Models: A Surgeon's Guide
/ Picture this: you’re about to deploy a shiny new 70-billion parameter language model, and your colleague asks the dreaded question: “How much GPU memory do we need?”.
- From Sequential to Parallel: Your Journey into GPU Programming with Triton
/ We all know that GPU programming is hard.
- The Transformer's Anatomy: A Deep Dive into the Architecture that Revolutionized Machine Learning
/ In the vast landscape of machine learning, few architectures have captured the imagination and transformed the field as profoundly as the Transformer. Like a master anatomist approaching a complex organism, we must carefully dissect each component to understand how this remarkable architecture breathes life into modern AI systems.
- The Machine Learning Surgeon's Guide to Quantization: Precision Cuts for Smarter Models
/ As humans, we perceive space and time as a seamless, continuous flow.
- The Operating Room Setup
/ Undoubtedly, one of the most critical aspects of machine learning is understanding the theory—without grasping how machines learn, you’ll never excel as an ML Surgeon!
- Dissecting torch.compile: Surgical Precision in PyTorch Optimization
/ You can take a look at the GitHub repository of this blogpost at this link
- A quick incision: ten minutes to RAG
/ In under 10 minutes, you’ll discover what RAG is, how to build a prototype for it in Python, and—most importantly—the true weight of the infamous Mr. Fat Raccoon.
- Performing Kernel Surgery: Profiling CUDA Kernels with NVIDIA Nsight Compute
/ Being a Machine Learning Surgeon is not an easy life.
- A Machine Learning Surgeon’s Toolkit: Advanced Matrix Multiplication in CUDA
/ If you want to learn how to write a CUDA kernel for matrix multiplication, look no further!
- Cerebral Cortex and Hippocampus: Understanding the Computational and Memory Design of GPUs
/ Just as a surgeon needs to understand anatomy, knowing GPU architecture is key to writing efficient kernels. You don’t need to be a hardware expert—just grasp the basics to unlock their full potential
- Hello CUDA: A Surgical Dissection
/ CUDA enables developers to harness NVIDIA GPUs for general-purpose tasks. This article guides you to a "Hello, World!" program as a starting point.
- An Introduction to Sparsity for Efficient Neural Network Inference
/ Sparsity is a solution to reduce the number of parameters and number of operations in Neural Networks, granting outstanding computational speedups and memory savings during inference.
mlsys2
- What Kimi K3 asks of the hardware
/ Reading the Kimi K3 announcement from the systems side. How linear attention changes the KV-cache budget, what 896 experts mean for the interconnect, and why the model ships in MXFP4.
- GRPO is mostly a systems problem
/ The GRPO config is small. The hard part is keeping rollout workers, trainers, rewards, and policy versions in the same story.
moe1
- What Kimi K3 asks of the hardware
/ Reading the Kimi K3 announcement from the systems side. How linear attention changes the KV-cache budget, what 896 experts mean for the interconnect, and why the model ships in MXFP4.
optimizers1
- A fused Lion optimizer kernel with Hugging Face kernels
/ I wrote a fused Lion optimizer step with Hugging Face kernels, packaged it with kernel-builder, and tested it on CUDA and Metal/MPS.
parallelism3
- Pipeline Parallelism: Surgery on Models Too Big for the Operating Table
/ In our latest article, we explored Data Parallelism.
- Data Parallelism: Scaling LLM Training Through Parallel Processing
/ In my latest article , I discussed the theoretical memory usage needed for inference and training with LLMs, highlighting the memory cost of each component involved in the process.
- The Memory Anatomy of Large Language Models: A Surgeon's Guide
/ Picture this: you’re about to deploy a shiny new 70-billion parameter language model, and your colleague asks the dreaded question: “How much GPU memory do we need?”.
pruning1
- An Introduction to Sparsity for Efficient Neural Network Inference
/ Sparsity is a solution to reduce the number of parameters and number of operations in Neural Networks, granting outstanding computational speedups and memory savings during inference.
pytorch2
- A fused Lion optimizer kernel with Hugging Face kernels
/ I wrote a fused Lion optimizer step with Hugging Face kernels, packaged it with kernel-builder, and tested it on CUDA and Metal/MPS.
- Dissecting torch.compile: Surgical Precision in PyTorch Optimization
/ You can take a look at the GitHub repository of this blogpost at this link
quantization3
- GGUF quantization, bit by bit
/ Q8_0 to IQ1_S and everything in between. Block layouts, scale hierarchies, codebooks, the importance matrix, and the mixed recipes used by llama.cpp, Unsloth, and DwarfStar.
- TurboQuant: What 3-Bit KV Caches Actually Mean for Your Inference Stack
/ A quick look to Google's new quantization method, with a bit of drama
- The Machine Learning Surgeon's Guide to Quantization: Precision Cuts for Smarter Models
/ As humans, we perceive space and time as a seamless, continuous flow.
rl2
- GRPO is mostly a systems problem
/ The GRPO config is small. The hard part is keeping rollout workers, trainers, rewards, and policy versions in the same story.
- The Memory Anatomy of Large Language Models: A Surgeon's Guide
/ Picture this: you’re about to deploy a shiny new 70-billion parameter language model, and your colleague asks the dreaded question: “How much GPU memory do we need?”.
scaling3
- TurboQuant: What 3-Bit KV Caches Actually Mean for Your Inference Stack
/ A quick look to Google's new quantization method, with a bit of drama
- Pipeline Parallelism: Surgery on Models Too Big for the Operating Table
/ In our latest article, we explored Data Parallelism.
- Data Parallelism: Scaling LLM Training Through Parallel Processing
/ In my latest article , I discussed the theoretical memory usage needed for inference and training with LLMs, highlighting the memory cost of each component involved in the process.
sparsity1
- An Introduction to Sparsity for Efficient Neural Network Inference
/ Sparsity is a solution to reduce the number of parameters and number of operations in Neural Networks, granting outstanding computational speedups and memory savings during inference.
strategy1
- Move Fast or Die Slow
/ Today’s article steps back from our usual technical deep-dives to examine the strategic importance of ML optimization.
systems1
- GRPO is mostly a systems problem
/ The GRPO config is small. The hard part is keeping rollout workers, trainers, rewards, and policy versions in the same story.
training3
- Transformer Attention, Backwards
/ You already know what a language model does. Let's reverse-engineer how.
- Pipeline Parallelism: Surgery on Models Too Big for the Operating Table
/ In our latest article, we explored Data Parallelism.
- The Memory Anatomy of Large Language Models: A Surgeon's Guide
/ Picture this: you’re about to deploy a shiny new 70-billion parameter language model, and your colleague asks the dreaded question: “How much GPU memory do we need?”.
transformers1
- The Transformer's Anatomy: A Deep Dive into the Architecture that Revolutionized Machine Learning
/ In the vast landscape of machine learning, few architectures have captured the imagination and transformed the field as profoundly as the Transformer. Like a master anatomist approaching a complex organism, we must carefully dissect each component to understand how this remarkable architecture breathes life into modern AI systems.
triton1
- From Sequential to Parallel: Your Journey into GPU Programming with Triton
/ We all know that GPU programming is hard.