Hollywood’s Future Lies Beyond Basic AI Prompts
Despite all the buzz around generative AI potentially transforming the filmmaking landscape, there haven’t been many projects that truly resonate as must-see entertainment. Most AI video models still struggle to produce anything beyond short, visually inconsistent clips. Some of the high-profile partnerships in Hollywood have fizzled out, suggesting that studios may not be able to fully trust the tech emerging from Silicon Valley. Currently, it seems that major production houses are only capable of churning out low-quality short-form content with generative AI. However, this could shift if studios take inspiration from some of the innovative projects showcased at this year’s Tribeca Film Festival.
While it seems unlikely that generative AI will ever create a fully engaging film from scratch, Tribeca featured several films that illustrated how human creators can effectively harness this technology. None of the AI-driven films screened at the festival were as disappointing as the low-quality content produced by some AI companies, but they did highlight why generative content often lacks the vibrancy of human-made art. For instance, “Roar,” an animated short from Illuminai Studios, felt more like a disjointed collection of AI-generated clips than a unified cinematic experience. Similarly, “ChikaBOOM!” from Asteria Film Co. missed the visual and auditory finesse needed to captivate audiences in its fast-paced tale of a magician in training.
The rough edges of “Roar” and “ChikaBOOM!” seemed to stem from the technological limitations inherent in their AI-driven production processes. In contrast, other films like Google DeepMind’s “Dear Upstairs Neighbors” and OpenAI’s “Mauvais Soleil” demonstrated how filmmakers could sidestep these issues by employing generative AI with greater creativity.
“Dear Upstairs Neighbors,” written and directed by Pixar veteran Connie Qin He in collaboration with Google DeepMind researchers, follows a weary young woman named Ada (Márcia Mayer, who also produced the short) as she struggles to find peace amidst the noise from her upstairs neighbors. The film’s unique style was achieved with the help of Pixar production designer Yingzong Xin, who created concept art using both digital and traditional methods. This painterly aesthetic posed a challenge for DeepMind’s researchers, as most AI video generation models would find it difficult to produce visually consistent footage in that style. However, DeepMind’s engineers crafted custom versions of their models to allow artists to fine-tune their outputs effectively.
By training these customized models on Xin’s artwork, they could consistently generate shots that aligned with He’s vision. The text-to-video models excelled at capturing specific stylistic elements, such as the visualization of sound during interactions. To ensure that “Dear Upstairs Neighbors” told a cohesive story, the creative team also relied on traditional animation techniques using Autodesk Maya, allowing them to control the unfolding of scenes precisely. Feeding these rough animations into the AI models enabled the artists to create visually refined scenes, enhanced with additional stylized assets.
More than any other film at Tribeca, “Dear Upstairs Neighbors” exemplified how generative AI can serve as a tailored tool that genuinely supports artists in their creative processes. The film’s workflow was heavily reliant on human artistry and the nuanced decisions that AI generators cannot make independently. It’s crucial to recognize that “Dear Upstairs Neighbors” would not have achieved its visual impact if produced with standard versions of Google’s models. The models worked effectively for this project, but that’s expected given its promotional nature for Google’s technology.
In comparison, “Dear Upstairs Neighbors” was a far more engaging experience than the films presented by OpenAI at the festival. Alice Gu’s semi-autobiographical drama “Smoked” utilized Sora to recreate the Palisades Fire, while Youssef Michraf’s “Mauvais Soleil” featured several photorealistic scenes generated with OpenAI’s tools. Both films showcased the filmmakers’ efforts to navigate the typical limitations of generative AI. For instance, wide shots in “Smoked” appeared somewhat cartoonish, though close-ups—shot using a Volume-like setup—of a woman and her son escaping the fire were more effective. “Mauvais Soleil” featured brief shots, with the only dialogue coming from an unseen narrator, but the story about a man’s life being distorted by AI made these choices feel intentional.
OpenAI’s presence at Tribeca was unexpected, especially following the company’s recent decision to shut down Sora entirely. This abrupt closure prevented OpenAI’s feature-length film “Critterz” from debuting at this year’s Cannes Film Festival. It appears that OpenAI may be shifting away from video-focused applications of its technology, but other players in the generative AI space are still developing tools that filmmakers can use to bring their visions to life.
With just $2,000 spent on computing costs, writer/director Ash Koosha managed to produce “Dreams of Violets,” a docudrama about the protests in Iran over the past year. Using tools like Kling AI, Claude, Gemini, and Nano Banana, Koosha crafted a fictional narrative about individuals trapped in an alley as police patrol the streets. The project took him only a few weeks to complete, and while it has a strong narrative foundation, it doesn’t push any visual boundaries.
After watching these films, it became clear that studios are unlikely to produce commercially viable projects solely by feeding prompts into generative AI models. While such content may persist, it’s not the type of work that major studios would want to associate with. A more probable future involves larger AI companies like Google collaborating with studios to create custom models tailored to specific workflows, which will only thrive under the guidance of human artists with clear creative visions.
