Friday , 19 April 2024
Home AI: Technology, News & Trends OpenAI Unveils New Model Sora: Generates 60-Second Videos From Text!

OpenAI Unveils New Model Sora: Generates 60-Second Videos From Text!

71
OpenAI sora
Sora can generate 60-second videos based on text instructions, capable of creating complex scenes with multiple characters, specific types of movements, and precise details of themes and backgrounds. It can also create multiple shots within a single generated video, accurately retaining characters and visual styles.

OpenAI is teaching AI to understand and simulate the physical world of motion, aiming to train models to “help people solve problems that require real-world interaction.” However, Sora sometimes confuses left and right in spatial details and struggles to understand specific instances of cause and effect.

Mammoths walk across a snowy grassland
Several mammoths walk across a snowy grassland, their long fur gently swaying in the wind, with snowy trees and mountains in the distance

After the successful development of the popular chatbot ChatGPT, OpenAI continues to innovate in generative AI. On February 16th, OpenAI introduced a new AI model, Sora, which generates “realistic” and “imaginative” 60-second videos based on quick text prompts.

OpenAI states that Sora can generate videos up to 60 seconds long while maintaining visual quality and adhering to user prompts. Sora can create complex scenes with multiple characters, specific types of movements, and precise details of themes and backgrounds. The model deeply understands language, accurately grasping prompts to generate convincing characters. Sora can also create multiple shots within a single generated video, accurately preserving characters and visual styles.

A cat wakes up its sleeping owner
A cat wakes up its sleeping owner

“This model not only understands what users are asking for in the prompts but also understands how these things exist in the real world.” OpenAI is teaching artificial intelligence to understand and simulate the physical world of motion, aiming to train models to “help people solve problems that require real-world interaction.”

In addition to generating videos solely from text prompts, the model can also animate existing static images, accurately animating the content, or extend and fill in missing frames from existing videos.

A man is running backward on a treadmill
Sometimes, Sora generates unreasonable actions, such as running backward on a treadmill.

However, Sora is still in development and has apparent “weaknesses,” particularly in spatial details where it confuses left and right, and it struggles to understand specific instances of cause and effect. For example, it might create a video where someone takes a bite of a cookie, but there are no bite marks on the cookie afterward.

Young wolves playfully chase each other
Five young wolves playfully chase each other on a gravel road surrounded by grass. A drawback of Sora is that animals or people may appear out of nowhere, especially in scenes with many entities.

In videos generated by Sora, animals or people may appear out of nowhere, particularly in scenes containing numerous entities. Additionally, Sora struggles to accurately describe events that occur over time, such as tracking specific camera trajectories.

A plastic chair is discovered in the desert
A plastic chair is discovered in the desert

Videos generated by Sora: A regular plastic chair is discovered in the desert, with people carefully digging and brushing off sand. In this example, Sora did not model the chair as a rigid object, resulting in inaccurate physical interactions.

In terms of model safety considerations, OpenAI plans to collaborate with an expert team to test the latest models, closely monitoring for errors, hate speech, and bias, among other factors. OpenAI also states that it is developing tools to detect misleading information, such as classifiers that can determine when a video was generated by Sora. Its text classifier can review and reject text input prompts that violate usage policies, such as extreme violence, sexual content, hate imagery, celebrity portraits, etc. “We have also developed robust image classifiers to review each frame of every generated video to help ensure that it complies with our usage policies before being displayed to users.”

OpenAI states that Sora will initially be provided to cybersecurity professors to assess the product’s harm or risks. Some visual artists, designers, and filmmakers will also gain access to Sora to collect feedback on how creative professionals use it.

A robot's life story in a "cyberpunk" setting
A robot’s life story in a “cyberpunk” setting

Reece Hayden, Senior Analyst at market research firm ABI Research, stated that while multimodal large models are not new, and models for text-to-video generation already exist, OpenAI’s claim that Sora stands out due to its length and accuracy. Hayden believes that such AI models could have a significant impact on the digital entertainment market, with new personalized content being disseminated across various channels. “A clear use case is television, creating short scenes to support storytelling.”

Related Articles

ChatGPT openAI

US AI Giants Accused of Secretly “Seizing” Data

The rapid development of artificial intelligence (AI) relies heavily on training models....

AI search for extraterrestrial life

AI Expands New Horizons in Search for Extraterrestrial Life

In the vast expanse of the universe, are humans the only known...

AI

“Will AI Surpass ‘Smartest Human’ in the Future? Experts Discuss Balancing Development with Fairness”

Elon Musk, the well-known American entrepreneur, recently sparked attention with his bold...

MTIA chip

Meta Launches New Version of In-house AI Chip: Performance Boosted Threefold

Amid a scarcity of AI chips, more and more tech giants are...