Inside the AI Tech Making Clones of Hollywood’s Biggest Actors | WSJ
AI tech is scanning the bodies, faces and even voices of some of Hollywood’s biggest actors to create digital doubles that can be transformed, de-aged, and even act in their stead. CAA, which represents actors like Tom Hanks and Zendaya, is embracing the tech that can clone their talent.
WSJ takes an exclusive look at the technology behind one of the most controversial flashpoints in Hollywood, and why the industry’s top talent agency is going all in on AI.
Chapters:
0:00 Digital doubles
0:50 Full body scan
1:57 Stretches
2:13 Why now?
3:54 Face scan
5:43 Voice scan
6:23 Post-production
News Explainers
Some days the high-speed news cycle can bring more questions than answers. WSJ’s news explainers break down the day's biggest stories into bite-size pieces to help you make sense of the news.
#AI #Tech #WSJ
The title is misleading. Photogrammetry is not "AI Tech." It's been around for over a decade. This is all relatively low-tech stuff but sped up with modern computing power.
Hi, Robert . your work is top tier
I think the "AI Tech" being referenced in the video is what studios are doing with the data, generatively. For example, voice cloning and deepfakes. The generative AI used for these applications originates from around 2014 but the mainstream use of GANs didn't come until around 2022 with the release of DALL E, Midjourney, etc. But I assume by misleading you mean the video doesn't do a good job differentiating what's new and old tech. People might think the photogrammetry aspect is the new tech, which it isn't.
@@user-sb5dc6eq4z It's like making a video all about this newfangled concept of "writing", framing it as something enabled by ChatGPT.
Who said the AI part was talking about Photogrammetry? AI is definitely needed for creating the Voice synthesizing but may also be aiding in the morphing of the 3D model and Shading / Coloring of the Skin.
I love he’s response, “no my wife keeps me very grounded 😂
😂😂😂😂😂
This practice has been standard procedure for 10 years in feature film making and has no connection to AI under normal circumstances. Scans that are digitized are animated using motion capture technology for the most part and are cleaned up using hand animation in the CG realm.
Yeah pretty funny hearing AI so often when this is something we’ve been doing for a decade now nothing to do with AI
@@AllanMcKay RIGHT?! you should show them your A.I explosions! That will quite literally blow their socks off! :)
@@davidedwardsme haha cheers mate :)
The generative AI tech they mentioned (only actually in a couple of the steps if you noticed), will just serve as an extra layer on top of the foundation tech that Currently exists. AI also greatly speeds up the process and will certainly improve it. The only part of this that is actually new AI tech would be the voice cloning which wasn’t possible before. Also, I think the most important part which they didn’t even mention, is the fact that as AI gets better, so does your “double”. All you have to do is run the data through a new model to generate an improved version without the need to go through the physical process again.
This video is super dumbed down for the population it's intended for (west) that's why lack of proper terminology like texture projection, rigging and all
why would they even need faces of familiar actors, they could generate good looking actors that do not exist
That's coming Next ... To avoid Copyrights 😂
Right now people have developed trust and a bond with the current persona's. The next generation will be the ones that adopt and are given these fake ai's as their trusted news and influence. Like a new computer code (os system), phase out the old methods while the young adopt the new programming. Only a great reset would allow for a drastic reality change like that of a completely new computer operating system.
That was a big sticking point in the actors' and writers' negotiation. They wanted human protection on work. I don't believe they won much of what they wanted.
Thats a question for the future(maybe somewhat near), right now you cant convincingly replace an actor for an entire movie. This is mostly used for replacing an actor in certain shots like face swapping for stunts. Gemini man is probably the most sophisticated example of a consistent digital double, but even then its much easier when this is based on a real person, especially when that person himself is filming a reference.
Because known actors have branding power. It's equivalent to branded products vs no brand products.
I think another issue that is very important to point out is the use of images, scans, and models of actors as training data without their consent to generate people that do not exist. By doing this, studios can get away with not paying residuals because they may argue that the generated doubles don't look like the actors. This is what happened to graphic and concept artists whose works were taken online without their consent and used as training data for AI image generators.
exactly , Revind, they always present this as "oh look what we can do". this aprticular technology they have been using it since 2004 and i know this because it was highlighted in the 3d world mag from summer 2004 and it was THE only reason i took down all my family pics online. all those matchovers, all the displacement maps that they will replicate mash up and generate iterations and variations that do not exist . all for free so that they can sell it back to other studios since there is no legal framework as of yet that prevents them from doing so. the fact that no legal frame work is mature to take control of this since the mid 2005 before this new age of Ai to me its a ploy to steal from all of us workers and even steal our likeliness in the process
can't apply old/current laws to new tech. AI is disruptive in nature. We need to adapt to the new challenges. To me it sounds like actors want to make money from royalties every time people even remember them in some movie. Pay artists well and move on. Royalties are a forever tax that shouldn't exist!!
I hope they use computer to replace terrible actors. Ai doesn't exist btw
The problem is the individualization of capitalism. AI is not stealing individuals, is stealing our entire culture. You can suppress Picasso of an AI data set and reproduce all its art, because Picasso is the result of a style of other classic European artists combined with African art. There are a Picasso recipe that is not the work of the individual Picasso. And it is the result of centuries of collective culture. Individual artists are irrelevant if you consider the entire history of our culture. And this is the weakness AI is exploiting.
@@edgarmondragon4708 AI ain't real kid.
Ethics and morals are not attributes I would assign to any talent agency, let alone CAA.
Or tech bros
How do ethics and morals come into this at all? No one is deceiving anyone, and you’re not just giving yourself away to a studio, you own everything in the process
Not gonna lie but the reporter looks like sam altman which makes this scarier💀
“Do you feel like you’re living in the future” “No…. My wife keeps me grounded” *Sigh* 😂
4:17 That is one profound statement. He knows that in the future the spoiled whiny women will be replaced by a perfect android partner. Wives, your days are numbered.
@@benplus2053You’d miss us. As Nietzsche said, The true man wants two things: danger and play. For that reason he wants woman, as the most dangerous plaything.
Thats the same camera rig as in the movie The Congress 😮
Just imagine in the future when everyone is digitally scanned. And then super intelligent AI can make that digital model do anything. Every politician, actor, friend, is controlled by the super intelligence. I cant imagine what sort of future it will be. I guess we are about to find out.
That makes no sense. Why would everyone be digitally scanned?
Future 🔮 is Amazon store and ever aspect of life will be controlled that means when you wake up when you sleep when you go to toilet when and what you speak and eat. All actions monitored we are heading to a AI GOD. If any confusion WAKE UP! 😮
@@cosmo9390possible in authoritarian governments. China is already doing. They have scanned the identities of all their citizens for the purpose of gathering data
@@cosmo9390 For OP's cockamamie dystopian future scenario to play out, they need to be. Because reasons.
@@fireaza 2007: “There’s no chance that the iPhone is going to get any significant market share.” - Steve Ballmer, Microsoft CEO.
As a Software Engineer, i never thought i would say it but, Technology has gone to Far.
As a fellow software engineer, I agree.
and this is just the beginning
This reminds me of bojack horse man,when he won awards due to his digital double performance
"They can then make a digital rendering of me". Weeks of work by talented and usually underpaid artists. This is by no means a trivial or automated process but heaven forbid we say that.
And what's going to happen when this 'vault' is hacked? 😒
You gonna have a bad time
dat Black Mirror reference
CIA Vault 8
Nothing. The artists could just as well put this as open source. They get paid whenever this is used in a siginificant commercial project, which by their nature is also an easy target coz it is a profitable target if the law is on your side.
You watched 8 minutes and didn't get anything. Sure it wasn't explained but what do you think the business modell is here for the scanning company?
If they do that to a significant fractions of actors, then there will be an anthropometric database a generative AI could be trained on, as StyleGAN was on the CelebrityHQ database, and then the studios will be able to generate actors as many virtual actors as they want. What the compensation for the real actors will be then is not easy to predict.
Can probably use the photos provided in this video to make a photogrametry of him.
Can't wait
So, do you have to redo the process every 10-15 years as you get older??
Wow can't wait for Mission impossible 152 😅
5:41 It's interesting to see how it can handle the catch light in your eye, (on a sunset scene for example). Right now even if your skin matches the environment, the light in your eyes is still a grid of LEDs, breaking the illusion in closeup shots.
I don’t think anyone watching a show/movie would expect to see a perfect reflection of the environment off the actor’s eyeballs. 😂 And as long as the lighting is close enough, that would do the job. Plus, I’m sure the developers had already thought of this and have required the AI to make those eye adjustments.
Bro they just digitally replace them, it's not that hard.
True. Consider those lighting scenarios as data that would be weighted based on novel environment lighting; i.e they'd ultimately be mixed to form something specific for any creative environment. The eyes would definitely need to be treated separately for the reasons you spotted.
You can make out the choppy frame drops in these 3D models...
Reminds me of the Al Pacino movie, Simone. Great movie
The sound engineer looks so stoked 😂
I could see some cash-strapped starlet selling the rights to her digital double for a paltry amount (like $50,000) while her digital double goes onto make millions for the studio.
It's like any other contract - be careful what you sign up for.
Dude that’s a no for me….
I liken this to back when man was creating the first cannon, or the first gun. It eventually was going to happen, but you wanted to be the first to do it so you could defend yourself, instead of waiting for someone else to make it and attack you with it.
defend ourselves from what, exactly?
I think it's interesting that they're positioning it at as a "Vault" rather than an exploitative commercial venture. We've just finished working on a Ryuichi Sakamoto holographic concert ("Kagami") which acts as a kind of 'performance vault' for his final public performance before his death earlier this year. It's a fascinating topic...
@@techcafe0the first guy with the gun. He is pretty much describing an arms race and what oppenheimer was about
The recurring pattern of the 21st century: don’t write new stories just remake old ones; don’t find new great actors just remake old ones.
Its going to be really interesting. Most human made movies suck but AI has a big potential to make every movie a great experience.
Sounds like Adversarial Neural Nets, with loads of computer vision algorithms processing industrial scale images.
IMHO the solution to who owns the Identity is clear... The primary holder should always be the Actor who can lease it to a Studio for the purpose of either making a Movie to begin with or finish it in case the case of the Actor's death which isn't exactly unheard of.
You looked like an alien AI by the end of the video. Haha
Dude, this technology is literally 20 years old and relays heavily on many artists. This is not AI. It's like claiming that photogrammetry of chrisps is AI.
Idk why but the face scan part gave me strong OA vibes haha
- Do you feel like you're living in the future? - No, my wife keeps me very grounded. lol
4:16 That was hilarious. We know who.wears the pants, lol.
Surely there would be some sort of short/long term effect [on someone] from the consistent flashing?! 🤔
Can they do makeup and plastic augmentation on the originalavatar?
This isn't AI .. this is purely what people unknowingly hate on ('CGI') .. or VFX. And just a chunk of the entire workflow and industry.
I think there can be a real benefit for actors. Imagine a star on his or her height of fame. You digitalize your body - and when you get too old to do action or chick flicks, you start selling your digital double to studios. Of course, you can still do age appropriate movies, but you can still cash in for what made you famous. This, of course, requires a copyright on your likeness - which will of course expire just like ordinary works of art. In essence, it can extend your market value and career for decades. Doesn't seem like a bad deal - given that legislation is modified accordingly, of course.
I hope we get to see Cristopher Reeves as Superman again one day with this tech.
Fascinating! Is anybody aware of this sort of technology in Spain? Oh, and Merry Christmas!
Erich Schwartzel(WSJ) was never in this video... Prove me wrong.
Digi doubles were used in films for over 25 years ...
can this be used to unlock the Face ID in your iPhone😢
A big no-no unless the actors are paid really well including royalties etc. Also give them the final say of the to be published media which includes their AI double
Good luck to the studios trying to import a 3D scan of an actor that was made 20 years ago into the current version of their animation software!
Images are images so they're pretty safe. It'll be the recipe that interprets the images that'll change almost monthly. We do this sort of stuff and it's understood that there's a raw capture and then the processing stage could be bespoke for the intended use. This video sort of glosses over the whole thing; but suffice that at least you have a capture that can be preserved forever and any interpretations would be time sensitive.
at that point they'd just throw it at an A.I and tell it to sort it out...
This Is The First Step, To STEP FORD PEOPLE!!!
For those interested in a Hollywood vision of where this might go, watch "The Congress". Robin Wright is really great, she plays a fictionalized version of herself doing this modeling.
This is terrifying. Imagine what political powers will do with this
Imagine a world leader dies and a shadow group uses this technology to Puppet the dead leader and no one would ever know.
What exactly would they do with it?
@@cosmo9390 They will probably go to sleep and let AI run the country
Deep Fake is quite old now and has been capable of lots of sensational possibilities. This process is massively more complicated, yet the possibilities remain unfulfilled...
@@trappedcat3615 That probably would be good for everyone
This guy looks Frankie Muniz LOL
What the WSJ is showing in this video is not AI, it's Machine Learning and VFX (CGI for the mainstream). It's been around for almost 20 years. Let's not confuse CGI with Ai, please! CGI requires large teams of Artists! AI does not!
Ha. He's funny. This was intereting.
I dunno if I wanna watch CGI actors though, I guess maybe for nostalgia purposes after many years have passed or something but I think it'll get stale fast unless the tech improves greatly.
This technology has nothing to do with AI and has been used in the VFX industry for over a decade. It is used to scan actors for a specific role in a movie and passed along to various HUMAN VFX artists who help animate and integrate the character into the film. Digitizing actors has been used for even longer than this technology. Look at the early Harry Potter movies during the quidditch scenes. Most often it is a CG Daniel Radcliffe. Or Spider-Man 1 and 2 in the early 2000s. The list goes on. Please stop with the fear mongering by creating video titles like these. It makes the WSJ look embarrassing as journalists when your information is wrong and discredits the talented VFX artists who help craft these incredible scenes in movies. If you want to have a discussion about ethics, that is completely valid; but understand what you're talking about before publishing media and spreading misinformation.
Where is the flashing lights warning? Video starts with flashing lights :D
"Some sort of ethics and morals" What sort?
Wow this is not dangerous at all
how artificial intelligence will take job of human actors
Most technologically advancements render many jobs obsolete, then new jobs are created. The invention of the internet, electricity, music streaming, the steam engine, and the internal combustion engine caused millions of jobs to be eliminated yet here we are today with just 3.7% unemployment in the U.S..
Huge Black Mirror vibes
Could the AI voice tech be used to speak in a language other than English if trained only in English? It would be extraordinary if an English-speaking American actor’s voice could be used for the dubbing for a different language, especially non-European languages.
This was foreshadowed in the movie The Congress- they knew it was coming!!
'No my wife keeps me very grounded.' No truer words have been spoken 😂 On another note, I'm looking forward to seeing this technology in video games. Seems the only person on it right now is Hideo Kojima.
Why do I feel like this might be the new Olan Mills but for the low low price of $1500 will scan your loved one, sample their voice and create a digital replica of them? Available in all malls as early as Spring 2025.
7:39 “Some sort of ethics and morals”
Don't worry, what could possibly go wrong?
Interesting.
There go the neighborhood 🤷🤦🏃✌️
He already looks like AI-generated through the whole video
“I could eat a peach for hours.” Yes I’m old 😆🤷🏼♂️
This isn’t ai. We’ve been doing this in vfx for YEARS. Someone just threw on the trendy “AI” term.
“They should have sent a poet.” 🤣
Black mirror is reel 🤣
It’s not called a clone anymore. It’s called a avatar ketchup, ketchup I have one too. We all have one.
this is not ai its hard work that many artist work on give them credit
Guys this isn't AI. This is a standard VFX workflow used for the past 10-20 years. Im tired of this fear-mongering calling everything AI. They brushed over the fact that this process costs tons of money and requires many many vfx artists over a long period to achieve. Journalists need to start actually doing some journalism I guess.
Reminds me of the “Joan is Awful” episode of Black Mirror, S6.
A future with out human. Where the simple expression of an emotion is emulated by a machine. A future where empathy is a lie and even when escaping from the world to watch a film brings you back to the paradoxical rollercoaster you thought you where escaping in the first place.
Eww this isn't really my type of stuff. I want to be organic.
Future will be new synthetic actors over real life actors and if people themselves can create their own synthetic actors then movie actors have officially lost the battle.
"Make my double do all other scenes but the kissing ones".
Please scan all the actors who are very old before they...
Don't forget about politicians, Putin's digital twin when combined with some general intelligence AI can run for reelection forever. Who can compete with the experiences of a few hundred years old men when today's pension-age presidents already winning due to advantage in decades of experience?
There are many reasons to use this tech. For cases where an actor has died before finishing shooting all the scenes or had an accident like Paul Walker or Brandon Lee, to sickness like Bruce Willis' aphasia and Michael J. Fox's parkinson. These are legit case uses. Actors or their families could get royalties from the use of the likeness for any project from movies to music videos, video games, TV ads, etc.
Another reason is being able to generate the "star" that will never go on strike. Win-win 😂
Those are not good reasons. Hire new people and stop giving all the same people all the money
I wanna see actors paying for the process from their pocket if they want to own it.
I’m not an actor but I most certainly do see a need to create legitimate laws to prevent people from being mistreated and or misrepresented. For example if someone is of a certain faith then that would prohibit someone from participating in certain actions. Therefore there shouldn’t be a digital portrayal of someone committing acts that go against the core beliefs of said person. There have been numerous people that have turned down roles due to the nature of the content being produced. In other words no one should have the ability to create essentially deep fakes of people saying or doing things that are not true or goes against that person’s core beliefs. That would be defamatory in nature to create false narratives with technology, especially if it is broadcasted to the public without the explicit consent or control of the persons being portrayed.
bro just got caught in 32K
😂🔥
One day everyone will wake up and realize that Carl Theodor Dreyer was the greatest filmmaker who ever existed.
Anyone who refuses to keep up with technology is already too old, regardless of their actual age. Throughout history, it's always been the elderly (60+) who refused to accept and embrace whatever new advances of their era were. It's actually a sign of old age to reject the new. So, if you can keep up with the latest technology of today, you can keep yourself young at heart. In my lifetime I saw personal computers, the internet, mobile phones and social media. I tell my 15 year old son that even though I talk about the 80's a lot, doesn't mean I want to go back there. The past is best remembered and not lived. I would hate to wake up and there be no internet. I cant wait to see what the future will bring. Don't grow old. Don't be scared of AI.
I already had enough after they make the gun shootings CGI 😂. We could never go back to the time when a guy from hongkong named jacky chan does some insane analog stunts bye bye.
Wow
I don't think they should be able to copy other humans without permission or a royalty if they are famous. Also All AI should be clearly branded AI so we can tell.
Spam calls want you to argue with them long enough for them to scan and synthesize your voice. Can TSA scan and duplicate your body?
Digi Doubles are now A.I? Wow .. im missing out on this tech! Can someone please show me where i can get me some A.I digi doubles? It would make my job a whole lot easier! .. My digi doubles, they just lay around in a T pose all day doing nothing, i really need to inject a little A.I wonder juice into my digi doubles and boost their work ethics.
data security? yeah.
The CEO guy looks very Mark Zuckerberge some how.
With Sora, this is already obsolete.
At least they are cloning real fresh and blood actors. When Hatsune Mikus become the norm then it's gonna be red alert.
One can easily achieve 10% of this quality today using just a smartphone and computer.
Where is the other side of the EQUATION???? You have AI taking all jobs but would subscriptions still be at a cost to "consumers". Other side of the equation being that "consumers" now have no disposable INCOME. So the entire capitalist consumerism norm has to also evolve. The words Capitalism, Socialism, Welfare, Tax, Income, Luxury are starting to take on a different meaning
He already looks like an artificial character.
Good one 👍
i prefer the ai actors personally - bring back bob hope for some fresh musty air we all need - he will be smarter and sexier and funnier than the original, he will sue bob hope and help the war effort
One day acting will be a team effort. You have the hot people that can’t act but are good at being fit and hot and taking pictures. Then you have the people that can act in a very wide range that wear the digital skins of the hot people while wearing motion capture gear to act out the scenes on green screens and digital background screens. One day it will cost almost nothing to make a good movie with the right number of voice actors and skins. Some people will have their skin bought and sold around Hollywood forever. Some actors will be in movies after they die like Carrie Fisher was in Rogue One as young Leia. Or like Peter Cushing as Tarkin after he died. I expect it will be the technology that decides whether the industry will have to change.