InfraLens

Lab 06: FFN / SwiGLU Parameter Count

This folder contains annotated reading material for the lab page.

Reading focus

Use code-like formulas to see why FFN/SwiGLU often dominates Transformer parameter count.

Source of truth

Running code is optional. The expected outcome is that you can explain the mechanism, the relevant state/shape, the common misunderstanding, and the interview answer.