Deep Learning

Layer Normalisation is a core component of modern Transformer architectures. This article explains normalization fundamentals, internal covariate shift, why batch normalization fails in self-attention, and how layer normalization works mathematically inside Transformers—step by step with clear examples.

Self-attention models lack an inherent sense of word order. This article explains positional encoding in Transformers from first principles, showing how sine–cosine functions encode absolute and relative positions efficiently and enable sequence understanding.

Multi-head attention addresses a key limitation of self-attention by enabling Transformers to capture multiple semantic perspectives simultaneously. This article explains the intuition, working mechanism, dimensional flow, and original Transformer implementation of multi-head attention using clear examples and mathematical reasoning.

This blog explains why self-attention qualifies as an attention mechanism and why the term “self” is used. By revisiting encoder–decoder attention, Luong attention, and alignment scores, we build a clear intuition for how self-attention works within a single sequence.