All Posts

A detailed, step-by-step explanation of how Transformer inference works, covering encoder outputs, autoregressive decoding, masked self-attention, cross-attention, and token-by-token generation with clear mathematical intuition.

A complete, step-by-step explanation of the Transformer decoder architecture, covering masked self-attention, cross-attention, feed-forward networks, and the final softmax output using an English-to-Hindi translation example.

Cross attention is a key mechanism in transformer encoder–decoder models that allows the decoder to focus on relevant parts of the input sequence. This guide explains cross attention step by step, compares it with self-attention, and shows how output representations are formed using input context.

Transformer decoders behave autoregressively during inference but allow parallel computation during training. This post explains why naive parallel self-attention causes data leakage and how masked self-attention solves this problem while preserving autoregressive behavior.

A clear, step-by-step explanation of the Transformer encoder architecture, covering tokenization, positional encoding, self-attention, feed-forward networks, residual connections, and why multiple encoder blocks are used.

Layer Normalisation is a core component of modern Transformer architectures. This article explains normalization fundamentals, internal covariate shift, why batch normalization fails in self-attention, and how layer normalization works mathematically inside Transformers—step by step with clear examples.

Self-attention models lack an inherent sense of word order. This article explains positional encoding in Transformers from first principles, showing how sine–cosine functions encode absolute and relative positions efficiently and enable sequence understanding.

Multi-head attention addresses a key limitation of self-attention by enabling Transformers to capture multiple semantic perspectives simultaneously. This article explains the intuition, working mechanism, dimensional flow, and original Transformer implementation of multi-head attention using clear examples and mathematical reasoning.

This blog explains why self-attention qualifies as an attention mechanism and why the term “self” is used. By revisiting encoder–decoder attention, Luong attention, and alignment scores, we build a clear intuition for how self-attention works within a single sequence.

A complete technical breakdown of Fast R-CNN and Faster R-CNN, covering RoI Pooling, quantization effects, Region Proposal Networks, anchor boxes, IoU labeling, multi-task loss, and why replacing Selective Search with RPN transformed object detection into a fully end-to-end trainable two-stage architecture.