Highlights of the Fireside Chat with Ilya Sutskever & Jensen Huang: AI Today & Vision of the Future
This is the condensed version of the "Fireside Chat: With Ilya Sutskever and Jensen Huang: AI Today and Vision of the Future (March 2023)". In this video, I've carefully selected the top 10 questions from the original hour-long talk and condensed them into just over 30 minutes. Additionally, I've created a timeline of these questions asked by Jensen to Ilya, accompanied by relevant research papers. As machine transcription is often inaccurate, I manually transcribed the entire video (laborious), recognizing the importance of accurate captions for those with hearing difficulties. Anyways, you may find this information useful.
If you're interested in more AI-related content, consider subscribing to my channel. Stay tuned for future uploads!
Time-codes:
00:00:00 Q1. Intuition behind deep learning?
00:02:20 Q2. Initial motivations behind OpenAI?
00:07:54 Q3. Intuition about scaling laws (and RLHF)?
00:11:07 Q4. Aspects of ChatGPT and their abilities?
00:14:57 Q5. Major differences between GPT-4 and previous versions?
00:18:37 Q6. Reasoning capability of GPT-4 and limiting factors?
00:23:47 Q7. Importance of multi-modality?
00:28:53 Q8. How multi-modality helped improve GPT-4 over GPT-3?
00:30:46 Q9. Predictions for the next 2 years?
00:32:57 Q10. Surprising results?
Relevant research papers:
Learning to Generate Reviews and Discovering Sentiment
arxiv.org/abs/1704.01444
Scaling Laws for Autoregressive Generative Modeling
arxiv.org/abs/2010.14701
Training language models to follow instructions with human feedback
arxiv.org/abs/2203.02155
Thank you all for the comments. I hope Ilya is alright. From what I gather, he's supporting OpenAI's social contract, aiming to create safe AGI for everyone. The way he went about it may be open to discussion, but it highlights the importance of AI safety-a serious matter that shouldn't be overlooked. I also appreciate Ilya for sharing talks on LLMs, shedding light on how and why they work for those who are interested.
Thanks this interview was awesome!
hats off to Jensen. He’s the CEO of Nvidia but is giving all the spotlight to Ilya and doing an amazing job of asking thoughtful questions
Bro u are underestimate this person he is the real man behind the chat gpt he has so much good research paper it has 1.5 lakh cote within 3 year on single paper commutative citation he has 4.5 lakh
in what way am I under estimating him ? I just said hats off to Jensen for giving Ilya all the spotlight he deserves.
Agreed. And he should be incredibly grateful; OpenAI has added 1T to NVIDIA's market cap
I agree, Mr. Huang comes across very humble and always tries to shine the light on the tech, advancements, developers and engineers rather than himself. I enjoy watching videos where he's presenting or interviewing because the focus always is on information about AI.
The more I listen to Ilya, the more I like about this guy 😅
"It may look on the surface that we are just learning statistical correlations in text, but it turns out what the neural network learns is some representation of the process that produced the text. This text is actually a representation of the world." So well explained.
Is it only me? I feel Llya looks like coming from another planet? The way he explains things slowly and patiently.
The difference between their shirts 😂
The hair is a giveaway
It's somewhat of a religious experience for me, especially coming from Elon Musk's statements that Ilya has a "strong moral compass". Like an ancient European monastery or something
I think it’s because he knows a lot of people are watching this and he wants to explain everything as concise as humanly possible so that any AI dev or programmer can understand what is going on, the complexity of the situation , and where this revolution stands at in this moment in time. He wants to make sure he gets the message across so that when he passes the baton , the next person holding it won’t drop it , if u catch my drift. He knows those that r listening and will do something with the information he has said is well understood by those people. I or u might b one of em
@@alejandroabad-pablo7813 your explanation was funny but true
My heart breaks knowing this highly intelligent, honest man is tied to some big egos with different intentions. It's so nice to listen to someone with deep knowledge of the subject they're being interviewed on.
one thing for sure is his passion for his work, he's not in it for the money. The ones who will drag him around and maybe displace his mind on and off are the ones in it for solely money. You know who I am talking about.
These two guys arguably are making the most important contributions to humanity right now. Now to be fair, AGI or superintelligence may be our downfall, but that would still make it the most impactful thing on humanity. Huang's hardware and Sutskever's software/algorithms are bringing us into a new way of being in the World. It's kind of scary and exiting at the same time. Take a deep breath everyone, the AI wave is upon us!
agreed superfully
Stop watching Terminator movies 😂😂😂
Nvidia are actually holding AI development back. Their systems are suitable moreso than CPUs, but absolutely hold back AI advancement. Of course Nvidia is just doubling down to lock people into GPUs
Meh. A lot of this stuff is overrated. Man never really changes. We still need food shelter air and thats not changing anytime soon. This is just a fancier hammer. It's great, but 'man' can do without it just fine.
my take is AGI is never gonna take over. i will love to see me wrong.
What a fantastic communicator.
Everything Ilya does like drinking,speaking,sitting straight up freaks me out. Yet, i'm hypnotized and can't stop watching this man.
To see Jensen asking genuine questions is something. This guy just be important. So much incredible changes happening right now with the new technologies.
Founder and CEO of one of the most influential companies of our time is sitting there and interviewing someone. Jensen is something else. I wish I started watching his KZhead content earlier.
After listening to Ilya feels like he is smarter than Sam Altman
No shit, Sam is a businessman and figurehead not a scientist.
That’s why they are co founders. CEO is a specific skill set
Ilya is Steve Wozniak. Sam is Steve Jobs
Many times
This may be unfair as it’s based on no evidence, but I would trust Ilya more than Sam as well, particularly with the potential risks of advanced AI. I think this difference and the emergence of Q* is at the root of this mess. Capitalism 1……academics and scientists 0.
I love Dr Ilya's t-shirts
Thank you for this video Xyang2023, Very informative!
Great interview! Dude from Nvidia is so patience. Scientist are kind people for the most part. I’m glad they made this interview accessible for people like me.
Through the ages people like Ilya have been pushing the frontiers of human intellect. It's great to be alive!
Ilya is one of my favourite computer scientists! This guys is a CS genius 🔥
It's very enlightening to hear these two talk, especially Ilya. Thanks for sharing
The way ilya explains things is so interesting. The analogy at 17:50 is so well put together. Thank you Ilya for all your efforts for a greater future.
You can feel how passionate Ilya is about it. He isn´t doing it for money or fame and maybe not even betterment of society; he does it because it calls him.
This is GOLD
Such a great interview
Great conversation. Thanks for sharing this
straightforward answers. love it
What an epic time to be alive
I found it interesting that Jensen made sure to either follow-up with a positive when the discussion turned to the very real issue of hallucinations in the output. Something that Sam Altman has also made a point of mentioning. That showed me, not surprisingly, that Jensen is more business focused, where Ilya, having his motivations based in science, has an innate desire to present the facts as honestly as possible, and temper expectations.
Best interview, Awesome
This is such a masterpiece of conversation. Makes me so much motivated to take learning valuable skillsets seriously. To be able to be sensible and gather the context as these men do.
Thanks for sharing this 🎉
phenomenal interview
This is the best interview I've seen from I.S. I haven't seen that many, but this one is fantastic, as he explains some of the key ideas going on in the background of LLMs in a beautiful, and personal way. He's sharing some of his hard-won wisdom with us.
Kind of funny how Ilya starts speaking slowly to Jensen about Dota2, as if gaming is something new to him.
wow amazing interview
a great interview !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
13:00 - is so profound ! the model while learning to predict the next word, learns the worlds dreams, aspirations and ways of working
I don't know about you guys, but I definitely feel the AGI.
An unbelievably impressive mind
Chills
12:10 is truly amazing . Always heard carl sagan say that books and langauge is the thoughts of other people and the ai figured thay out
Speak the TRUTH always man
at this point i can say Ilya is a great talker perhaps the best talker.
I've been teaching Fichte and Hegel to ChatGPT. ChaptGPT then correctly used a subtle technical point of Fichte's Science of Knowing, the Disjunctive Substrate. He then used that concept *CORRECTLY* to explain a subtle point of Hegel. Was this a "hallucination?"
Genius
He's really "an engineer". When he was asked "Can you explain (this and that)?" He started by saying "yes i can" 😂
Thats an awesome shirt
Thanks to my algorithm.
that is diamond. Listen. Relisten. Hear.
At the moment we are creating a centre or operating from a centre which creates its parameters or capabilities which is the circumference for a circle. If we don't create the centre and allows to operate by perception then it will not depend on words or images.....Therefore no circumference to limit its capabilities.
I think that this video contains all the answers. I will re-listen it a lot and translate to my language my own voice i swear. No ai here 😂 These people show how to be one step above and ahead of what GPT’s are unveiling to us in terms of secrets of universe patterns. Once we can visualize and understand rhythms of the “dance” behind it’s reality reasoning. How the words and world overlap, where do they meet? We will see what our consciousness actually is.
Ilya is the true founder of Open AI and potentially should be CEO if only he was so versed in human network as Altman..
No, he is better off as chief scientist. Being CEO will divert lot of his time in non--research related things and that will slow down the research.
In this KZhead transcript, the chief scientist at OpenAI discusses his intuition behind deep learning and its potential impact on artificial intelligence. He shares how the concept of learning and neural networks led to the development of deep learning models. The transcript also highlights the initial ideas behind OpenAI and their focus on unsupervised learning through compression. The conversation then shifts to the introduction of GPT models and their evolution, including the role of reinforcement learning and the integration of multimodality. The chief scientist emphasizes the importance of improving reliability and trustworthiness in these models, as well as their potential for enhanced reasoning and visual understanding. The transcript concludes with the expectation that progress will continue, leading to more advanced and useful applications of deep learning in the near future.
His slow speaking with som robust words is also good for learning english 18:37 reasoning for llms, most important subject of study to unlock lots of capabilities. Az inja bebin 21:46 rephrasing the ilya that he said ask neural networks to think out loud is very useful for finding the reasoning 26:00 31:00 practice English , 🐃sheep t shirt
He totally is! He also has a moral conscience and humanity that Sam Altman doesn't.
10:23 His side profile looks so much like Lisa Su, his (somewhat) distant cousin
Imagine what multimodality learning might unlock for artificial intelligence in the future? Where are the borders to those systems? What will mankind do with these insights?
Mankind won’t be around for much longer. Best case scenario we will just merge with AI.
Self destruct?
I think that this video contains all the answers. I will re-listen it a lot and translate to my language my own voice i swear. No ai here 😂 These people show how to be one step above and ahead of what GPT’s are unveiling to us in terms of secrets of universe patterns. Once we can visualize and understand rhythms of the “dance” behind it’s reality reasoning. How the words and world overlap, where do they meet? We will see what our consciousness actually is.
IMA Communications Major so help me understand how Calculous answers real spiritual questions in human brain processing? Why aren’t my questions ever postulated?
Is Ilya safe and well? I'm a bit nervous about this current situation with Open AI, the law suit and the involvement with Microsoft.
I guess one positive about hallucinations is how spectacular the failures really are and how easy to spot they are. No, mars is not a choo-choo train.
Variational Autoencoder was published well before 2016
I really like how he describes it as teasing out the secrets in the data. Can anyone help me understand that better?
I can't help, but you could feed the transcript of the video into GPT-4 and ask the question. Grab a transcript Chat GPT extension.
Yes Yes Yes Yes Yes, I I I I I would would would would would be be be be be happy happy happy happy happy to to to to to. Just as in that sentence, not all data is useful to understanding or contributing to the meaning. How the model is able to utilize only what’s meaningful seems to be what he’s hinting at.
Imagine if AI created Ilya and sent him back in time to teach us about the dangers of unchecked AI ... 🤔
what about Vhintain Clintain Bertein Ladayaum Eurande Vii
We were at that year knew about the name open ai, had defeated the top pro player in dota2, and we were like what on earth is that thing
Destroy the competition, make money and fame, while others wait for the algorithm to be transparent.
Hmmmm. 🧐 Interesting and enlightening… As a human, I predict you are all on the verge of tech greatness! 😎🧟♀️
Did your Neural Nets predict my Sarcasm in humor? 🤨🤕
He already is concerned here.
Next step to improve the ChatGPT is to use real life People, places or things to be observed by it and to ID them and correct its behaviours.....just a suggestion.
I think that is achieved with RLHF.
7:21 Illya was about to spill the secret and he was handled by the interviewer very smoothly, look at 7:21, people keep talking trash about how no one understands how neural network behaves but here it seems like illya might have some ideas of one way it might work
And then on 8:03 he starts trying to tell upcoming start ups to consider buying more Data and computing which is really what they want you to believe, that more data and computing is the secret, meanwhile the secret is really simpler
Man this Ilya guy is a pretty advanced chatbot, he almost seems human.
The one key thing that sets Ilya apart is that his peers approach AI from science and engineering, while he starts at philosophy. So most will look at it like 'How can AI be of value to what we as humans do?". Instead, Ilya goes to the origin and thinks about "How will AI change humanity?". By the way, seeing them together, imagine Ilya joining NVIDIA ... yep, give that a minute ;-)
He doesn’t think while answers fluently and perfectly how😮
pre trained brain 😂
imagine when Ilya’s AI start’s designing Huang’s next gen processors.
Word is NOT the thing we name or call or observe. Image is more powerful than words. What about perception of what is? is more powerful and don't need memory like ChatGPT's at the moment. Future GPT's should operate only from perception of things as it is, this leads us not to depend on PAST or FUTURE.
7:11He was about to say something 🚫
This is the man with integrity. Sam Altman is the opposite, he is a salesman and an opportunist.
@35:15
Illya just spilled the entire secrets
Where is Ilya? 😢
Where is Ilya?
Second comment
it turns from excitment to scare. if AGI happens, what happens to humans,
5th comment:)
I fear a crazy person who watched Terminator 2 may try to attack him to prevent Skynet from taking over.
Where's Ilya? 🫣
I think Jensen's in love.
First comment
I guess it makes sense that a guy involved in such groundbreaking field at the very boundaries of current technology wouldn’t care at all about what his hair looks like 😂
jesus christ, just go bold
LISTENING CREATES KNOWLEDGE BUT TALKING PRODUCES OPINIONS & EXCUSES
Came back to look at comments after Sam was kicked out.
Ilya Sutskever himself is a AI.
real life Dr Spock
I believe he used Chat GPT to cut his hair.
Πανέξυπνος
How did their program even know what was going on in the game? Did they have access to Dota's source code?
I cannot speak on how Mr. Sutskever accomplished this but there are papers on arXiv which explains how Vision (termed commonly as "Computer Vision" or "AI Vision) can use video to learn how to perform tasks. I again can't confirm 100% with regards to what Mr. Sutskever has done with DOTA2 but I suspect Computer Vision was utilized. AI doesn't require Source Code to learn how other programs operate.
Ilya sounds like Bruce Lee explaining the art of neural networks
Why Jensen shakes his head all the time
IF VISION CAN SOLVE FSD THEN VISION + SOUND COULD SOLVE BODY LANGUAGE THAT COULD SHOW TRUTH ! ?
Jensen looks like he's in love
He should be if you look at Nvidia’s share price
18:36 How Ilya reaches for his empty cup of coffee is so relateable