[{"_1":2,"_155":-5,"_156":-5},"loaderData",{"_3":4,"_10":11},"root",{"_5":-5,"_6":7},"user","ENV",{"_8":9},"GA_MEASUREMENT_ID","G-FZLJC5WJDF","routes/category.$slug",{"_12":13,"_79":80,"_88":82,"_89":90},"posts",[14],{"_15":16,"_17":18,"_19":20,"_21":22,"_23":24,"_25":26,"_27":28,"_29":30,"_31":32,"_33":34,"_35":36,"_37":38,"_39":-5,"_40":41,"_42":43,"_53":54,"_72":73,"_74":75,"_76":77,"_78":36},"documentId","s8lzta0zo13exrpjs8xhg02e","title","Alibaba Has Developed An AI-Generated Video Model \"EMO\"","slug","alibaba-has-developed-an-ai-generated-video-model-emo","excerpt","Alibaba recently launched a new AI image-audio-video model technology EMO, officially referred to as \"an expressive audio-driven portrait video generation framework\".","content","\n

After the United States' OpenAI launched Sora to ignite the internet, Chinese internet technology giant Alibaba Group Holding Limited is now stepping up its efforts to catch up.

\n\n\n\n

Alibaba Group Holding Limited's Intelligent Computing Research Institute recently launched a new AI image-audio-video model technology EMO, officially referred to as \"an expressive audio-driven portrait video generation framework\".

\n\n\n\n

It is reported that you only need to provide a photo and an audio file, EMO can generate AI videos that can speak and sing, as well as seamlessly integrate dynamic short videos, with a maximum length of about 1 minute and 30 seconds. The expressions are very accurate, any voice, any speed, and any image can be matched one by one.

\n\n\n\n

For example, in the TV drama \"The Knockout,\" \"Gao Qiqiang\" talks about Luo Xiang's law popularization; a picture of Cai Xukun can be used with other audio to \"sing out\" a rap song, even the lip movements are almost identical; and recently, in the video case of Sora released by OpenAI, an AI-generated Japanese street heroine wearing sunglasses can not only speak now but also sing beautiful songs.

\n\n\n\n

The Alibaba research team stated that EMO can generate sound avatar videos with rich facial expressions and various head poses, and it can also create videos of any duration based on the length of the input video.

\n\n\n\n

At the same time, EMO also has audio-driven portrait video generation, rich expression dynamic rendering, support for various head turning poses, increased dynamism and realism of videos, support for multiple languages and portrait styles, fast-paced synchronization, cross-actor performance transformation and other features and functions.

\n\n\n\n

On a technical level, researchers at Alibaba shared that the EMO framework uses the Audio2Video diffusion model to generate expressive portrait videos. This technology mainly consists of three stages: first is the initial stage of frame encoding, where ReferenceNet is used to extract features from reference images and motion frames; second is the diffusion process stage, where a pre-trained audio encoder processes audio embeddings. Facial region masks and multi-frame noise integration are used to control facial image generation; third is using the backbone network to facilitate denoising operations. In the backbone network, two forms are applied - reference attention and audio attention mechanisms, which are crucial for preserving character identity and adjusting character actions respectively. Additionally, EMO's temporal module is used to manipulate the time dimension and adjust movement speed.

\n\n\n\n

In fact, over the past year, Alibaba Group Holding Limited has continued to make efforts in AI, including launching a variety of large-scale AI products such as Qwen-VL that compete with OpenAI on Alibaba Cloud, as well as technologies like Outfit Anyone based on the dual-stream conditional diffusion model for human face transformation and Animate Anyone for character animation models, achieving applications in multiple scenarios.

\n\n\n\n

Currently, Alibaba is still assisting in the development of technology applications related to robots, digital humans, and Agents based on generative AI technology.

\n\n\n\n

In addition, Alibaba is currently one of the major technology companies in China's open source model field, creating and operating the Chinese AI model open source community 'ModelScope'. Since its launch a year ago, the number of model downloads on the 'ModelScope' community has exceeded 100 million. Earlier, Alibaba also released an all-in-one large-scale model service platform - Aliyun 'Bailian'.

\n\n\n\n

In addition to developing its own AI model technology products, Alibaba is also promoting investment in some large-scale AI companies.

\n\n\n\n

In February of this year, Alibaba led a new round of financing for the domestic AI large-scale model team MoonShot AI with $1 billion, pushing the company's valuation to as high as $25 billion and making it the largest single-round financing for a Chinese AI startup.

\n\n\n\n

Nowadays, Chinese tech giants such as Alibaba and Tencent have taken action to support early-stage AI startups in China through various investment methods, promoting the development of large-scale AI models in China.

\n\n\n\n

According to research firm CB Insights, in 2023, China's investment in the field of AI was about 232 deals, a year-on-year decrease of 38%. The total financing amount during the same period was approximately $2 billion, a decrease of 70% compared to the previous year.

\n","publishedDate","2024-02-29T15:47:43.000Z","featured",false,"viewCount",0,"readingTime",4,"status","draft","language","en","originalWordPressId",163896,"featuredImage","featuredImageUrl","https://assets.pandaily.com/uploads/2024/02/EMO.jpg","categories",[44],{"_15":45,"_46":47,"_19":48,"_49":50,"_51":52},"ou7tkn9cq6eba9jr68hqiwch","name","AI","ai","color","#000000","description","","tags",[55,60,64,68],{"_15":56,"_46":57,"_19":58,"_49":59,"_51":52},"d93dhsf37ni748u8e3l4tsbt","Alibaba","alibaba","#6366f1",{"_15":61,"_46":62,"_19":63,"_49":59,"_51":52},"h192oah6gbqfbkqkwfzajq24","OpenAI","openai",{"_15":65,"_46":66,"_19":67,"_49":59,"_51":52},"d0jvwbhhh1dmkwn5y5vzq2rk","Sora","sora",{"_15":69,"_46":70,"_19":71,"_49":59,"_51":52},"udzw1ruvokupwm6ca68ts5if","EMO","emo","createdAt","2025-06-24T22:23:22.120Z","updatedAt","2025-06-24T23:44:36.034Z","publishedAt","2025-06-24T23:44:36.072Z","locale","pagination",{"_81":82,"_83":84,"_85":82,"_86":87},"page",22,"pageSize",12,"pageCount","total",253,"currentPage","categoryData",{"_15":45,"_46":47,"_19":48,"_49":50,"_51":52,"_91":30,"_39":-5,"_12":92,"_72":153,"_74":153,"_76":154,"_78":36},"order",[93,99,105,111,117,123,129,135,141,147],{"_15":94,"_17":95,"_19":96,"_21":97,"_39":-5,"_25":98},"ttdoniyzsp5zmpolhht0nun2","ByteDance Releases Technical Report on Seedream 2.0 Text-to-Image Model","bytedance-releases-technical-report-on-seedream-2-0-text-to-image-model","ByteDance's Douyin AI team published a technical report on their Seedream 2.0 text-to-image generation model on March 12, revealing comprehensive technical details for the first time.","2025-03-12T21:20:57.000Z",{"_15":100,"_17":101,"_19":102,"_21":103,"_39":-5,"_25":104},"enbmyf7iv0vc7hrgdy1ngynk","Manus Reaches Strategic Cooperation with the Alibaba Tongyi Qwen Team","manus-reaches-strategic-cooperation-with-the-alibaba-tongyi-qwen-team","Manus has reached a strategic cooperation with the Alibaba Tongyi Qwen team to achieve all functions of Manus on domestic models and computing power platforms.","2025-03-12T14:36:20.000Z",{"_15":106,"_17":107,"_19":108,"_21":109,"_39":-5,"_25":110},"njgly2y8srldl6tanrooaoj9","Alibaba Releases and Open-Sources QwQ-32B Reasoning Model, Challenging Deepseek R1","alibaba-releases-and-open-sources-qwq-32b-reasoning-model-challenging-deepseek-r1","Following the release and open-sourcing of Tongyi Qianwen's latest reasoning model, Alibaba's stock price surged. Overnight, U.S. stock markets closed with an 8.61% increase, closing at $141.03. ","2025-03-06T12:02:23.000Z",{"_15":112,"_17":113,"_19":114,"_21":115,"_39":-5,"_25":116},"vfg5nvyusyqk0r0noxm0p3j1","Douyin Integrates Doubao AI","douyin-integrates-doubao-ai","By directly integrating Doubao AI, Douyin can not only enhance its own AI capabilities but also introduce a broader user traffic to the latter. ","2025-03-05T16:58:35.000Z",{"_15":118,"_17":119,"_19":120,"_21":121,"_39":-5,"_25":122},"b84ylciqy6uvxyseffz74kzc","CogView4, An Open-Source Text-to-Image Model, Has Been Released by Zhipu AI","cogview4-an-open-source-text-to-image-model-has-been-released-by-zhipu-ai","On March 4th, Zhipu AI released the first open-source text-to-image model that supports generating Chinese characters - CogView4.","2025-03-05T10:27:36.000Z",{"_15":124,"_17":125,"_19":126,"_21":127,"_39":-5,"_25":128},"p7kxcws4r5fb4nl4qdzqbf1w","Surpassing DeepSeek, Tencent Yuanbao Rises to the Top of the Download Ranking","surpassing-deepseek-tencent-yuanbao-rises-to-the-top-of-the-download-ranking","Tencent Yuanbao App has risen from second place to first place in the free app download ranking on the China region Apple App Store, surpassing DeepSeek.","2025-03-04T14:21:13.000Z",{"_15":130,"_17":131,"_19":132,"_21":133,"_39":-5,"_25":134},"u3y1xo8qckqhiiqqv9y70zse","DeepSeek Discloses A Theoretical Cost-Profit Ratio of 545%","deepseek-discloses-a-theoretical-cost-profit-ratio-of-545","DeepSeek published an article on Zhihu, revealing for the first time the core optimization solution of the model reasoning system and disclosing a theoretical profit margin as high as 545%.","2025-03-03T17:05:06.000Z",{"_15":136,"_17":137,"_19":138,"_21":139,"_39":-5,"_25":140},"ibns0etb4m6ebsl1qupuvnid","Zhipu AI Secures Strategic Financing from Hangzhou State-Owned Investors","zhipu-ai-secures-strategic-financing-from-hangzhou-state-owned-investors","On March 3, Zhipu AI announced a strategic financing round exceeding 1 billion RMB, with participation from state-owned investors in Hangzhou.","2025-03-03T12:39:00.000Z",{"_15":142,"_17":143,"_19":144,"_21":145,"_39":-5,"_25":146},"ezo5pkvfi3ubfn12wybouloc","Tencent Hunyuan's New Generation Turbo S Fast Thinking Model Released","tencent-hunyuans-new-generation-turbo-s-fast-thinking-model-released","On February 27th, Tencent's Hunyuan new generation fast thinking model Turbo S was officially released.","2025-02-28T14:24:17.000Z",{"_15":148,"_17":149,"_19":150,"_21":151,"_39":-5,"_25":152},"k96c9x9e8imra1in73oazup5","Alibaba Open Sources Video Generation Model Wanxiang 2.1, Performance Surpasses Sora","alibaba-open-sources-video-generation-model-wanxiang-2-1-performance-surpasses-sora","On the evening of February 25th, Alibaba fully open-sourced its video generation model Wanxiang 2.1.","2025-02-27T12:44:27.000Z","2025-06-24T21:42:37.888Z","2025-06-24T21:42:37.886Z","actionData","errors"]

Category: AI

Alibaba Has Developed An AI-Generated Video Model "EMO"