Netflix-owned Eyeline, Scanline VFX’s virtual production division, is putting it out there with AI video in the form of Go-with-the-Flow, described as “… an easy and efficient way to control the motion patterns of video diffusion models. It lets a user decide how the camera and objects in a scene will move, and can even let you transfer motion patterns from one video to another.”
Go-with-the-Flow is currently in preview on Github with the animation of still images. According to the development team, “We simply fine-tune a base model — requiring no changes to the original pipeline or architecture, except: instead of using pure i.i.d. Gaussian noise, we use warped noise instead. Inference has exactly the same computational cost as running the base model.
Visit https://www.youtube.com/watch?v=IO3pbQpT5F8 and https://github.com/Eyeline-Research/Go-with-the-Flow