Coarse-to-Fine Lucas–Kanade

F vs. backward-warped G at every scale · in-scale Gauss–Newton EPE —

Pyramid · coarse → fine · F | warped-G per level

estimate ground truth increment Δ feature F / G in overlay

Run

show truth

flip active level

Inner-scale convergence

Mean step |Δ| per inner iteration. Dashed ×2 = level change. Bars shrink as Gauss–Newton converges within each scale.

Parameters

Pyramid levels 4

Window radius (px) 7

Inner iterations / level 4

Pyramid blur σ 1.0

True translation X 11

True translation Y 7

Swirl strength 3.0

Per level, per inner iteration:

backward-warp G by current w (sample G at x+w) → recompute Iₓ,I_y,I_t on the warped window → solve Δw=(AᵀA)⁻¹Aᵀy → w←w+Δw

. The middle panel is that warped G — watch it slide onto F. The overlay step superimposes them (F red, G cyan); colour fringing = remaining misalignment the next step removes. Crossing a level multiplies positions and vectors by 2. N next · Space play · R reset.
Texture: a frame from your Fusilli Jerry clip; the second frame is generated by a known flow so the green estimate can be checked against red ground truth.