We've seen plenty of weird examples of AI generated imagery, including Photoshop's stunning Generative Fill, capable of, as the name suggests, filling an image with an AI-generated background. Fewer examples of AI video have done the rounds (save for Will Smith eating spaghetti – link at the bottom if you're so inclined), but this version of generative fill for video is pretty incredible.
A Reddit user has shared a video showing how AI model StableDiffusion has been used to fill in the tops and bottoms of shots from the Harry Potter film series, creating the effect that they've been filmed in a vertical orientation – and it's far more authentic-looking than you'd expect. (Looking for more AI strangeness? Check out the weirdest art created with AI.)
using AI to fill the scenes vertically from r/StableDiffusion
The video shows how only a small fraction of the image is from the actual movie – the rest is AI generated. And from hospital rooms to the Hogwarts school hall, it seems StableDiffusion has no trouble generating believable sets from the film franchise.
Reddit users are impressed with the footage, but many have also pointed out potential limitations. "One of the most elegant implementations of AI I've seen when it comes to content. It works beautifully on these clips," one comments, "but I wonder how many types of scenes it doesn't work well with. I suspect there's a high variance between types of shots it aces or totally botches." "Yeah it will only work well on shots that are static/locked camera, or that can be tracked and stabilized, outpainted, then re-matchmoved," adds another. "And if people cross the edges of what's being outpainted, you'll have all the usual flickering and inconsistency problems."
Still, it's clear that when it works, it really works. And even in just a few short months, we've come a long way from that Will Smith spaghetti video. Not that we should be surprised – just look how much MidJourney has improved in one year.
