BLOG ILLUSTRATION / EXECUTION STORY

Train with generated history

Real past supplies input for teacher forcing. Student-generated history supplies input for the student. A teacher supplies supervision for the student’s distribution.

Train with generated historyReal past supplies input for teacher forcing. Student-generated history supplies input for the student. A teacher supplies supervision for the student’s distribution. TEACHER FORCING SELF-FORCING Real past ground-truth latents Student Condition with real past Training objective supervised prediction Generated past student rollout Student rollout Condition with past New latent student prediction Teacher loss
Figure 4. Self-Forcing trains with student-generated history and teacher supervision for the student’s distribution. Teacher forcing uses real past. Orange shows generated-state feedback, which alone cannot ensure long-horizon stability. The author made this diagram to show the relations. Open full-size SVG