MultiHeadAttention

Record fields

Name	Description
`this.forward (x, ?mask, ?kvCache)`

Pre-norm transformer block: Norm → Attn → Add → Norm → FFN → Add.

Name	Description
`this.forward (x, ?mask)`

Post-norm transformer block: Attn → Add → Norm → FFN → Add → Norm.

Name	Description
`this.forward (x, ?mask)`	$x \leftarrow \mathrm{LN}(x + \mathrm{Attn}(x))$ , then $x \leftarrow \mathrm{LN}(x + \mathrm{FFN}(x))$ .