Mark Zuckerberg - Llama 3, $10B Models, Caesar Augustus, & 1 GW Datacenters
Zuck on:
- Llama 3
- open sourcing towards AGI
- custom silicon, synthetic data, & energy constraints on scaling
- Caesar Augustus, intelligence explosion, bioweapons, $10b models, & much more
Enjoy!
Timestamps
00:00:00 Llama 3
00:09:15 Coding on path to AGI
00:26:07 Energy bottlenecks
00:34:03 Is AI the most important technology ever?
00:38:04 Dangers of open source
00:54:40 Caesar Augustus and metaverse
01:05:36 Open sourcing the $10b model & custom silicon
01:16:02 Zuck as CEO of Google+
Links
Apple Podcasts: podcasts.apple.com/us/podcast...
Spotify: open.spotify.com/episode/6Lbs...
Transcript: www.dwarkeshpatel.com/p/mark-...
Me on Twitter: / dwarkesh_sp
Sponsors
If you’re interested in advertising on the podcast, fill out this form: airtable.com/appxGOvFLDLP5dlz...
- This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue. Learn more at stripe.com/
- V7 Go is a tool to automate multimodal tasks using GenAI, reliably and at scale. Use code DWARKESH20 for 20% off on the pro plan. Learn more at www.v7labs.com/go?...
- CommandBar is an AI user assistant that any software product can embed to non-annoyingly assist, support, and unleash their users. Used by forward-thinking CX, product, growth, and marketing teams. Learn more at www.commandbar.com/
He's looking and acting extremely human here, must be running llama 3 under the hood.
Jokes a part, I like the new Zuck.
"I really like bbq" "I have my grill behind me here"
He actually became self conscious.
Zuck with the pivot of the century
He does Brazilian jiu-jitsu and smokes meat. Of course he is human.
Bro you made it if you're interviewing Zuck. Congrats homie.
Holy shit this channel has got to blow up now
And he got a good convo out of Zuck. Very insightful.
only down hill from here!
Big facts
And he watches and complimented the show
It looks like Mark might be the first person to actually beat autism
They beat it of out him (in a controlled manner). Just further proof bullying works.
Man u wont know how much i giggled at this comment lol
No. The *uhms and Ahs ahs ehm..!* Are strong with this one..
Rumor has it it was Theo Von, but Mark looks to have conquered a whole different type of 'tism. It’s scary good tbh
@@chynabad9804 It was probably just hanging out with MMA guys instead of silicon valley tech weirdos that drove that change
This was probably Mark's best interview yet.
It wasn't Mark. It was llama 3
human likeness is uncanny
Totally agree - that was awesome to watch. Never been a fan of him, but that insight into his thinking was incredible!
Really think so? Because imo there were too many "what if" and future questions, nobody can really answer. Nothing really gained from those questions. More interesting would have been why Llama-3 is better, what did they do for achieving it, does "open-source" for Meta also revealing their data sets etc.; I don't think this interview was so interesting.
@@thirdeye4654 Yeah, I agree the questions could ve been better
Enjoy folks :)
Bruhhhhh, Congrats, so glad i stumbled on your podcast, the most unique set of interviews I have ever seen. See ya when ya hit 1 million :)
How did you get Zuck on?
Holy crap nice job
actually the best podcast on AI imo
Shame you didn't ask him how he feels about having so much genocide blood on his hands? He is censoring anything from Palestine but not Israel. This guy is not ur friend.
a sincere 1 hour conversation with Mark Zuckster on my timeline? oh you know i'm sparking up for this one
Right up there with you!
same
but he is a psychopath
@@tubbyszabo6586 stop acting out like a woman and have some drugs
@@tubbyszabo6586 maybe first look up what that means
Mark's commitment to open-source deserves HUGE respect, thanks Mark.
Without massive hardware there’s only a limited amount of what you can do with an open source mode like this
It's not open-source..
@@futurecenterofficialyes it is
@@thems_the_brakes It's not only about LLMs, as he mentions also OCP for datacenters, but PyTorch and React too are Meta's creations
@@ActiveGamingUK it's not , have you read the fine print?
this is where you see zuck is also a psychology genius (and ready to do anything to achieve his goals). He asked himself what he should do to increase is influence. Look relatable. So he hired a rebranding/restyling coach, changed is way of speaking (more like a normal person), wears an oversized shirt, says he does sports, etc and now it's like it's not the same zuck. But the goal is the same. To be loved and make his company more influential, loved, etc. Among all tech billionaires, he is the most like this, ready to adapt to achieve his goals (learn new languages, skills, change appearance, etc)
On an interview with Lex I believe, he mentioned that he was into extreme sports, but he just sounded robotic. He sounds completely different and more natural here.
He is the most like this? Musk is obsessed with appearing relatable.
Zuck has gotten so much more personable the last few years. I am starting to really like him.
That's what happens when you focus on interpersonal skills versus solely engineering. Its very easy to get stuck in engineering mode and not know how to communicate with fellow human beings.
He got a new update
Is because his doing Jiu Jitsu, that humbles you really fast. OSS
no
I trust Z more than Sam Altman. A lot more.
What I love is how transparent this conversation, clearly no preparation done on Zuck’s end
Wake up babe, Dwarkesh just released another interview
Dwarkesh is really the best at interviewing these high calibre tech and historian guests. I feel like I connect with the ideas and information 3 times as much with his interview style, questions and proper pushback. Thank you for podcasting!!
You should have a look at Lex Fridman. 👍
@@Ben_D.Lex is monotonous!
@@Ben_D. he’s great aswell! As a budding academic I get a lot from him, but I find his podcasts aren’t as informative.
@@Ben_D.find Dwarkesh a lot more sophisticated in his questioning and very knowledgeable Lex goes a bit esoteric at times
I agree that he does a good job at this. I unsubscribed from Lex when all of his podcasts started getting several hours long, completely lost focus and became a meandering mess. I appreciate the more focused interview style.
he finally did the right concoction of ayahuasca, let go of that stiffness and blossomed into a socal bro. Good job!
I’ve gotta say, I’ve never really been a fan of zuck, but something about him over the past 9-12 months and made me respect him a hell of a lot more. It’s good to see him with that entrepreneurial spirit and fight in his eyes. Dudes grown on me.
Yeah he actually seems alive now. I can dig it.
@@Sephaos hes taking human classes
New PR team is doing a good job
Same, this seems like the first time in while where he looked actually excited about tech again
he is the only founder ceo remaining in silicon valley. everyone else has thrown in the towel
31:25 Interesting cut in the video at 31:42 , I'm very curious to know what that was about. But based on my calculations there's about a 54 to 61 second gap in that cut based on the cloud movements in the background. There's a lot that could have been said in that 60 seconds, wouldn't be surprised if someone off camera hopped in to tell them to cut, or Mark said something he didn't want to XD How I got that: The cut in the video has a cloud with a tip at 1719,202 (in 4k) before the cut, and 1689,189 after the cut. 80 seconds earlier at 30:22 the same cloud tip is visible at 1763,219. 44,17 difference is 80 seconds so 30,13 difference is ~54-61 seconds
touch grass bro
@@jb_kc__ Yeah you're probably right. Just one of the richest men in the world running one of the most powerful companies talking about society transforming technology that may end in the extinction of our lives and everything we know and have ever known. Probably silly to care about that, I should touch grass instead
@@thenautilator661Oh shit. You’re serious 😂. Cut your internet off for a bit buddy.
@@uncoiledfish2561 assuming you're not trolling, tell me why you think the cut in the video isn't worth my comment/time/interest
Seemed like Mark was caught off guard by the statement about Amazon's power plant and possibly said something a bit speculative/disparaging, or said something about their own plans that he doesn't want in the public space just yet.
Dwarkesh, this is one of the best tech interviews I've watched in a long time. Your ads are also brief and to the point-perfect!
I love this new version of Mark🔥🔥. Excellent pod!
did Fuckerberg get a updated firmware?
decades of pure evil disappeared overnight in a single podcast. I see now how humanity got to where we are at today.
On what version is he to date ?
@@growthcab Humanizing "Caeser" Zuck version/phase: Pax Romana
37:12 Great line 🙂 "I think that another bias that people tend to have is thinking that intelligence is somehow kind of fundamentally connected to life and it's not actually clear that it is"
I thought the same thing. This interview actually had a number of profound thoughts in it. Really a good listen.
@@tmmerlo yes 🙂
Really great interview Dwarkesh and Mark. I really enjoyed watching and following it through. It's not another antagonistic and sensation looking interview you see coming out everyday on internet. Just a casual professional conversation with very important questions getting asked and being answered in a very friendly and humane manner. Made my day. Thank you!
marks a fan that’s such a huge compliment if someone who has the means to listen or talk to pretty much anyone listens to u
Only thing that can be more exciting than the model releases is this interview. Thanks Dwarkesh! I hope you reach 100k subs this week!
What a legendary slip @17:10 for the CEO of a company in a two-sided market
He said the quiet part out loud..
More telling his is slip up just a few seconds later - "they're not going to want an AI that sells their competitor's customers... sorry... their competitor's products"
@@grigrob9he's driving the reality Wht d u mean
Yeah, for Zuck, the customer is the product.
I was hoping to find a comment mentioning it😂
One of the best interviewes of Zuck I've ever heard. Thanks for sharing!
We reached the singularity, the ZUCK24 is undistinguisheable from a real human being.
beautiful comment!
Can definitely tell that Mark has really settled into himself. At least on camera and in interview. He always seemed so guarded and rehearsed before.
He used to look like he was constantly held at gunpoint Like someone has their finger on the trigger, aiming right at his head while he tries not to stumble over his words
He was in front of congress…lol
A line of coke helps before interviews
Hes like 40 lol wtf
It’s engineered. His personality is purely to increase his company’s rate of influence. In other words his new "openness" seems to be semi deceptive. But not entirely.
So hyped to see you get top caliber guests, much deserved! 🔥
feel like he's been getting top caliber guests for a long time now but I mean yeah I agree in general this podcast is goated
God do I just love the Zuck. Best CEO in tech in my opinion, and all that while Meta / Facebook was one of the most hated companies on earth. Hats off to you Mark!
The host is asking real smart follow-up questions. This is very rarely the case in podcasts, it's so refreshing.
All of his podcasts sound well researched in advance. It's really great stuff
Got this Podcast as KZhead Recommendation and subscribed after listening to this conversation...Mark's View on AI was a different perspective and interesting as well. Something new to learn from this conversation
Bro didn't lie when he said it was not April's fool.
haha yes! I seriously thought it was when he tweeted it!
Really well done, I didn't expect this at all. So happy for you!
Zuck comes across so well here, like the most human I've seen him. He seems so at ease and open here. Well done @DwarkeshPatel on a great Interview
I was surprised by the honesty in this interview. "Is intelligence fundamentally connected to life?" Great topic!
You have solved the problem this channel had with poor audio. Super. Now it sounds natural and clear.
I think since Sheryl Sandberg is gone, he does not have a communication department telling him what to say, not to say ... he just goes with the flow and be open... fantastic interview
I have to say man, your tenacity, your interrogative style, which is often in the form of "ok yeah that's interesting now let's talk about something that doesn't exist yet, what do you think about that," when pushed to extracting Mark's self-reflection on his drive to always be building "something that I haven't solved yet" really really worked out. Kudos. I keep dialing into your pod because you just don't let up. awesome.
Gemini vs GPT is a battle. But, Gemma vs Llama is war. ✨
lets fucking goooo
Gemma is a joke. What are you talking about?
@@joaquindominguez3846 may be. Last i checked, it was a model
“Open source is the time between development and proprietary software” is such a funny way of putting it but so true!
congratulations fam! its been such a treat watching you grow!!
bro out here casually dropping some of the most goated podcast episodes! excited to see where you go mate!
congrats on llama 3 release! the timeline just got better ;)
Zuck seems to always try and be calculated on camera, and I’m glad to see him finally start being more relaxed and comfortable. Love this new light Mark ❤
Great interview as always! You and Lex are definitely my top interviewers at the moment
Please keep it up. Pretty excited about more on these sides of tech
17:14 Loved the confusion between customers and products there.
Can you explain what he meant there? Or what that unintentional slip was?
@@SanatanRevival he said "sell their competitors' customers"
Also, in the youtube subtitles „customers“ was deleted at 17:14
is it because customers are the product?
We’ve gone from Rogan to Lex and now, finally, there’s a podcaster who is more intelligent than us so that we can learn from two people instead of one. Its 100% more efficient and I like it. Congrats on getting Zuckerberg, its a crowning moment.
Sam Harris has been doing it for years
I’m loving Mark’s new energy!
Mark has finally shed his skin and now he's got an entirely new body for himself. The face looks perticularly more human now. 😮😮
updated software as well
@@DaTruthHeals no, not software. This time it's definitely AI. 😅
the ai that meta is working on is really helping him seem more human 😜
I think it's growth hormone or testosterone or something. But not to take anything away from him. He obviously works hard to stay in shape and also is contributing to open source in an amazing way.
That`s what software updates do.
This is an amazing interview - thank you!
I like this version of Zuck! It feels like his home environment, nerding out about tech in details.
Thanks for contributing to the economic rot and the preeminence of late stage capitalism by doing ads. Its highly appreciated
Dwarkesh and I just 7-8 years ago were eating ghost peppers on the back of a charter bus and now this man knows Zuckerberg. What am I doing with my life..
But that's a good thing right ? Your good friend knowing and interviewing Zuckerberg?
@@Notsosarcastic_02 it’s a great thing!
How did Zuck end up being the most normal and possibly the least evil one of these billionaire tech guys?
Least evil? IG is literally facilitating child predators.
Agreed
Meta ending up as the least evil company in AI is also not something I would have expected
What makes him the least evil in your opinion?
Are you high
He touched on one thing in particular for my own use case, which is an LLM hamstringing someone incorrectly-when I try to create an image based on a public domain figure like Peter Pan, the model still refuses to create it, even though it totally knows it’s public domain. This kind of stuff should be easy to fix, but these LLMs are black boxes, we aren’t coding them per se, they create their own rules sometimes, I don’t know.
You’ve grown faster in podcasting than Zuck did with social media! Congrats big guy! 👍
This interview is gonna positively change people’s perception of Zuck. Well done Dwarkesh
Zuck's sleeves are the most bizarre length
true
Uniqlo has some like that, oversized style, pretty laxed
Oversizedmate. I believe Zuck is pretty partial to gym-culture. Why not celebrate it?
When you're that rich you can afford the extra fabric and never have to suffer from cold elbows like a pleb.
It’s just oversized get with the trend grandad
Boshidan kuzatyapman, foydali bo’lyapti Raxmat Aziz aka!
voy oʻzbekcha? ajoyib
That interview with Sholto Douglas and Trenton Bricken was so very cool too
Zuck's really grown on me over the last year or so
Same. I used to despise the guy. I like that he’s been working on his fitness for example.
Somewhere, a pr person in Silicon Valley is smiling at your comment. Calculated effort.
@@boxed119 what, the fitness thing? Yeah I agree
@@socrates2788 who cares about his fitness. he just released the best opensource model in the world today.
@@boxed119Nope
"when our open source gets good enough, we won't keep it open" - Zuck
So happy to see this podcast getting the recognition it deserves. You just need to interview Sam Altman for the AI grand slam.
This interview felt REAL. Some REAL questions from Dwarkesh to Zuck. Sickkkk!
11:34 i think there’s some research now that shows improving coding performance also improves model reasoning and vice versa
Where was this filmed? the elevated walkways in background look awesome
Facebook office?
@@MarcoBonechi elevated walkways 😂😂
Probably mountain view
Glad to see the steady rise of this podcast
Great timing on the day of the Llama 3 release. Congrats, Dwarkesh!
The humanlike progression of mark is far outpacing the humanlike progression of AI! Mark made a really good case for open source here.
He has been doing a lot of RLHF
WHAT , dude u have made it . From now on I would regard u as an A-list podcast in the realms of JRE and lex . wow more of this impressive guest scouting and interviewing
First time seeing your pod, you did great and it's definitely a net plus for Mark's Image!
Hearing about Meta's approach to open sourcing infrastructure (more broadly than AI) was very interesting. Going by 01:05:36, it seems that Zuck himself is quite involved in that strategy.
There should have been a handshake at the end.. Amazing interview.
Crazy heel click at 55:30!
good spot haha, she must be rlly happy
so random lol
Zuck’s model of when open source is really interesting . When his product is best among all existing open source models and not as good as closed products and there is some loose thinking that open source can standardize some infra which helps reduce costs. No shits given to theoretical risks. I love it.
I think it's really great that you asked about and focused attention on having at least some kind of plan for (un)expected developments in model capabilities. It's fine to put a low probability on bad behaviours, but they should at least be able to recognize when those happen and have some idea what to do in that case.
Dwarkesh is the new Lex Fried men
Lex Fried Men 😭
@@ken_kaneki197 lmao
Way better
1000x better.
Lex "I work at MIT and build Robots, do jiu jitsu to infiltrate into the JRE world and get wett talking about Elon Mah" Fridman 🤡
17:14 "Sell their competitors customers" lol
Freudian slip 😂
Best interview with Zuck I have seen. Previously seen ones with Marques and Lex.
He does make a good comparision between Apple's full control of features and big corps deciding what you can build with AI.
46:50 "We need everyone to have guns, and the good guys will have stronger guns than the bad guys" Appreciate you challenging his assumptions here
Wow congratz, Zuck. Thats huge!
Dwarkesh is just so chill interviewing these owner of tech giants and working professionals in th software industry . Any other interviewer would just react like a child being excited but dwarkesh is just to calm and knows how to do his job . A Big applause !!
I think he is actually a corpo chill. It is nonetheless watch an hour long advertisement for the model i am already playing with.
@@claxvii177th6 yeah i figured it out that he might be from that field that's why he have the experience
Oh my fucking god my guy! Congrats on this interview, you're absolutely crushing it! SO hyped to see this!
Anthropic CEO ✔️ DeepMind CEO ✔️ Llama CEO ✔️ OpenAI’s real brain(Ilya sutskever)✔️
He's more personable and his questions are better than Lex. Hope he gets Sam or Elon at some point.
@@cagnazzo82Sam is excellent but for Elon I would be always sceptical.
Mark is really the good guy; people will realize it sooner or later. His approach has always been about empowering people, unlike a handful of other companies that try to monopolize and create barriers for the small guy.
You're just saying that cause he grew out his hair tho
Stealing and using our data for any reason he wants and you trust this guy? Facebook affecting elections, spreading lies, worsening mental health. This guy has made the world a worse place. Let's see what he does with AI though I doubt he has suddenly changed his tune. EDIT: When he says "Conviction and values". Values? Ha!
kind of obvious now in hindsight, absolutely attacked by both sides of MSM
What he's doing with Meta AI is certainly great, but he has created a lot of harm through his primary products over the years. Willingly prioritizing profit over the well-being of teenagers, even with internal reports showing the negative effects those products have. I think it's kind of naive to frame him as the good guy.
yup obvious in hindsight, both sides attacked him relentlessly
LFG!!!! you totally deserve this man your AI interviews have been some of the best on youtube.
Insane interview get just as Llama3 drops. Congrats.
Is it me or The Zuck is like living his second youth, he's only missing an ear expansion, he even has some weird rizz here. Guys don't be Fooled, AGI is already here, there's no way the Zuck looks this human without an Upgrade
Martial arts has been good for him
I think Facebook sucked his soul out by being a mature platform. It was not a passion project anymore, and it was just finding ways to exploit the user for more profit. AI is a new field that allowed him to find his interest again. Just go make things and see where it leads you.
is zuck actually slowly becoming human
Has his nose returned to normal size? Who is Geppetto though?
Amazing! Can't wait to listen !
Great podcast. Love hearing Mark's views on his vision for AI and the fact he's foreseeing the dangers ahead.
Best interviewer ever! Legit knows more than all of AI podcasts bros.
As of this writing, Llama 3 is not open source. It specifically prohibits use, commercial and otherwise, in certain conditions: github.com/meta-llama/llama3/blob/main/LICENSE Meta has made the Llama 3 source available and has allowed use in certain conditions but the license is not open source, under the commonly held definition of open source: en.wikipedia.org/wiki/Open-source_software
They literally talk about that
@@marwin4348 I'm sorry but I can't find where they talk about how they haven't actually open sourced their model. Can you point to a timestamp?
@@abetusk kzhead.info/sun/lcdvpaqOcHWAep8/bejne.html
It's open as long as you aren't a 700m+ DAU platform. And, if you are, requiring they fork over share of revenue makes sense for meta.
@@ricopags That requirement makes it not open source.
31:40 WHAT DID HE SAY
Interesting cut
First time listener- GREAT episode!Subscribed, and I’ll be baaach.
Super cool interview. Thanks for creating this.
Man, doing sports really does wonders to disembodied nerdoids
Damn how do one bring such fancy guests ?
yeah i dont understand, who is this guy? how does he have this kind of access?
He is respected in tech
Good interview. It feels more risks and fear focussed than how open sourcing of these models would go along way
since you had Amodei on I knew this channel was the next big thing, congrats