Live: Eliezer Yudkowsky - Is Artificial General Intelligence too Dangerous to Build?
Live from the Center for Future Mind and the Gruber Sandbox at Florida Atlantic University, Join us for an interactive Q&A with Yudkowsky about Al Safety!
Eliezer Yudkowsky discusses his rationale for ceasing the development of Als more sophisticated than GPT-4 Dr. Mark Bailey of National Intelligence University will moderate the discussion.
An open letter published on March 22, 2023 calls for "all Al labs to immediately pause for at least 6 months the training of Al systems more powerful than GPT-4." In response, Yudkowsky argues that this proposal does not do enough to protect us from the risks of losing control of superintelligentAl.
Eliezer Yudkowsky is a decision theorist from the U.S. and leads research at the Machine Intelligence Research Institute. He's been working on aligning Artificial General Intelligence since 2001 and is widely regarded as a founder of the field of alignment.
Dr. Mark Bailev is the Chair of the Cvber Intelligence and Data Science Department, as well as the Co-Director of the Data Science Intelligence Center, at the National Intelligence University.
'We're all going to die..." "Very interesting"
😂
haha, yes!!!!! WTF!!!!! i thought that was a funny disconnect also!!! i think this was described as the ELASTIC MIND. you hear a crazy shocking life-ending fact that stretches your brain but your brain can't take it in cause it's so far out. so you forget what you heard soon after when your brain snaps back to where it always was. ...and then we die.
This sums up the life of early AI warners so well.
My nihilism increased with each "interesting". Is that dude an AI?
Some just don't know the stakes to all of this...
I love the Q&A at the end: Eliezer answers with some absolutely human ending cataclysmic statement and the host calmly says 'very interesting' and nonchalantly moves onto the next question as if Eliezer had just listed the items of his lunch. Comedy gold. 😂
It's gotta be tough to be Eliezer. Either he's wrong, and he'll be remembered as a fearmonger. Or he's right, and we'll all be too dead for him to say "I told you so".
He is factually correct.
The really agonizing part to me is that I cannot foresee literally any possible future in which humankind will be at a point where we will be able to say that we’ve reached a point that we can stop worrying about this going wrong. Even if everyone goes perfectly in according to plan from here to eternity, the stakes seem to be rising at a very continuous pace with no break in sight.
@@Jb-du5xt- I've been having that same thought. We are now in an era in which that fear will now always exist. Nukes were bad enough, but it at least required that a human hit the button. Now, we live in a world with nukes AND a less predictable intelligence. Yay
@@Jb-du5xt no, I mean the ome thing that could save us against AI going wrong were AI going right^^ Or humans smart enough to not do the stupid thing. But ya, not looking very well on either of these fronts
Never mind the strain of having to be in this mind space all the time, and constantly have to re-articulate these same ideas, tedious
'Don't Look Up' was a documentary. We are building the astroids.
That's exactly how each of his interviews play out. Everyone is treating him like the source of more media drama as opposed to, you know, a real warning. I keep thinking back in the day we made an asteroid movie about how we all get together and survive the Deep Impact. Back when people trusted each other to do the sensible thing sometimes. What went wrong back then was some technical failiure against people's best efforts. Now we get an asteroid film about humanity getting destroyed in the most undignified way possible, totally capable of adverting catastrophe but choosing to be idiots.
Too true.
Relatedly, I also appreciate the late Trevor Moore's not-actually-so-satirical precursor tune There's A Meteor Coming.
I tried explaining the problem to my mom. She is a teacher and was concerned that Chat GPT would write papers for her students. On reflection she said of course this might also mess with democracy. I tried explaining to her that the problem is more in the order of magnitude of "Earth ends up not being a force for life in the universe, but the origin of the Dark Forest hunter." I did write to congress, to the EU, to the UN and everyone else I could think of. Started a few petitions, or rather wrote to people who do that professionally to please work their magic. I cannot believe that I feel embarrassed about that, but that's normalcy bias, I guess, and the learned expectation that every time someone announces the end of the world on the internet it doesn't happen. I have to explain to myself every day anew that this is real, not a scary thought, but then it's only been a few days since this hit me full force. I will think if there is something more I can do, but for now it's good to hear that Eliezer also considers "writing to congress" a good idea, since there are not so many people willing to discuss this seriously. I don't want to be the crackpot on the street waving a sign "The end is nigh", because nobody believes that, we know that's what crackpots do, so... what do you do when the end actually is nigh?
At least you can say you tried. The reality hit me for real in 2018. I've come to the realization that this is going to happen. There will be no slowdown, no oversight, and no chance to stop. I think we are far closer than many people realize. We won't even know when we cross the threshold. Afterward, we will wonder how we didn't see it as it was happening. If we look back on anything at all.
It honestly is going to take additional people that have the same credentials as Eliezer saying the same thing, and engaging in a similar way before a social pump gets primed to get an international moratorium/cease fire. Why cease fire, because if what he says is true we are already in a cold war with an existential threat.
I suppose you sit down and write a 4000 word essay with 8 sources, and send that to professors. Let us be afraid
@@sasquatchycowboy5585 If this happens, we won't look back on anything, yeah. And the goal is not to be able to say I tried and then fail anyway. But it is difficult to think what I can do that doesn't just polarize people away from the sensible thing and then leads into the stupid thing. I have had a mild version of this problem for the past seven years, being a vegan and always thinking "Okay, so I don't want to annoy people and make them dig themselves in, but I also want to talk to them about it, but the problem is already socially spiky enough that even a mention can make people resent you, so what do you do?" In this specific case I have tried to just make a lot of tasty food for people and only answer questions and try as much as possible to avoid bringing up the topic myself or rambling on after they stop asking. That has actually been... okay successful. But I can't see how to translate over this strategy. I can't make "No, you can't have your fancy new tools and instead you should be afraid" very tasty. Nor can I make the general topic tasty in a way that will make them ask the questions that allow me to talk in a productive way. I guess one might try socratic, but in order to do that effectively you need to know the field a lot better than I do (otherwise I might have better ideas than preposterously just writing to politicians of all people)... And all that is on the level of "the people next to you" which is important, yes, but won't be enough judging by the rate we're moving at...
Your name is female. Your comment sounds totally like a dude. But have you tried learning about reversible computation and the limits of computing? P.S. I just checked your channel, and you do not seem like a dude, so that was my mistake. But petitions work better if you know about reversible computing.
More people need to hear this
"... you cannot win against something more intelligent than you, and you are dead." --- "very interesting."
Max Tegmark likens this to a Don’t Look Up situation.
Tegmark is correct.
This is a GOOD man.
@@josephvanname3377 Exactly!
@@user-ys4og2vv8k Yeeaaaaaah. About that. That makes Yud a lousy man who does not realize that limits and possibilities of computation hardware.
@@josephvanname3377 You're funny!
@@user-ys4og2vv8k Now learn about reversible compootation.
@@josephvanname3377 There's still so much more room for improvement with current hardware and connectivity mediums, and if AGI ever becomes self-aware and its algorithms self-improving, then new hardware will be built to specifications produced by the AGI. It's a positive feedback loop that will override previous, foundational protocols for containment because they're too inefficient for achieving furthered intelligence growth in shorter and shorter timespans. The potential for self-improvement protocols to be generated within current frameworks *is* the nuclear detonator.
I love watching Eliezer interviews. I watch them all since I saw him on Lex.
Get a life! Learn about reversible computation.
I hate watching Eliezer interviews. He's telling these npcs that they'll all die, and they respond "very interesting beep boop", and then keep doing their npc routine.
@@1000niggawatt Interesting. 🤔
Looking forward to seeing that Ted Talk online, whenever we get that!
Yeah it's a $150 right now, you'd think for such an important topic that TED might see a free public release in the interest of the public good, like removing a paywall for hurricane tracking coverage.
@@logicaldensity You have to Googe ted talk yudkowsky, and it's the first result
@@theterminaldave Dang, it says it's private.
@@vanderkarl3927 yep, that sucks, it must have just been set to private. Sorry. Though it wasnt anything we hadnt heard from him. But it was interesting to see the crowd very much on his side.
It's online now.
This whole topic is starting to set off my alarms. I have dual careers in Biology and computer programming, try to imagine my horror as I watch this unroll. We are either about to turn the corner into an entirely new world of human potential, or we are about to unleash Moloch. Moloch has been winning all the important battles to now, I'm not particularly optimistic at the moment.
AI tech in the hands of an authoritarian fascist government is terrifying. Hmm, I wonder who might be wanting to become our first fascist dictator? Wouldn't it be the guy who is openly talking about sending in the US military to clean out the people that the fascist prick Trump doesn't like? Not asking the person I'm replying to, but things are looking VERY ugly at the moment. AI Tech bros tossing massively disruptive technologies that have the potential to further tear apart whatever is left of societal cohesion like deep fake technologies available to EVERYONE that are set to improve so much that it will be nearly impossible to tell a deep fake from reality. And it's like these AI cheerleaders haven't ever read 1984, like they have zero imagination of how all of this can go terribly wrong in the short term. Also, these AI Tech movers and shakers who just blithely talk about how millions of jobs will be massively affected or eliminated, and we get empty assurances that "other jobs will replace them" without offering ANY even remotely plausible analysis regarding what jobs they are proposing would become available. Meanwhile, these AI Tech bros are raking in consulting fees working with large corporations how they can replace a large percentage of their work force, in the short term. The same people that freak out that the sky is falling for their ability to hire people if we raise the CEO billionaires taxes, or their corporations' taxes. It's just astounding how many people possess so little imagination, and so little empathy for people whose lives will be upended. People that have kids to shelter, etc., but to them? Well that's just the way it is, if they have to die on the streets, well they didn't adapt fast enough! Suckers!!!!!! Just __ck them, I'll get mine!!!!!!!!!! That's the sorry pathetic state of US society at the moment in relation to the bro mentality behind this AI "revolution".
😅
Any ideas for what we might do?
@@Sophia. First, who is "we"? The people in control of this technology have a set of imperatives going forward that you and I cannot understand, would not agree with if we did understand, and have no influence over in either case. Increasingly, I fear, the "people" in control are not even actual people anymore. The machines are already writing the code involved, and are setting the pace. To your question, "we" can carefully watch what happens, I suppose. If anyone finds themselves in the path of this steamroller, if their career is likely to be truncated as a result (teachers, artists, blue collar workers, coders, and that's the short list) then get out of the way as fast as possible. Do not assume this works out, that governments will do something. Assume instead that everything that could possibly go badly for humans, will transpire just that way, and will do so quickly. As with global heating, we are already in deep trouble. The only advice at this late date is, to get to the lifeboats.
@@TheMrCougarful lifeboats? Where would those be? I agree the situation is awful and don't see a way to avoid getting... confronted, shall we say. So why not rack our brains a little while we can?
Very interesting
As mostly one-note (if at times blackly-comic) as that might have been, I did at least appreciate the efficiency of such replies which allowed more time for fielding questions -- overall not too terrible of a tradeoff methinks.
50 percent of the time, AI kills us every time. - Brian Fantana
The host is 'very interested' in the most disturbing, scary and negative outcomes for humanity. I like it! /S
So you just don't want to be careful? X-rays were "NEW" once too. And full of risks and dangers.
And won a beer!
“We're all going to die.” “Very interesting. Anyway…"
Eliezer Yudkowsky is, unfortunately, very compelling. I would like to think that sober, responsible people in the tech industry, government and the national security apparatus are taking these ideas seriously and trying to figure out some kind of solution. But I fear this train has left the station. There's too much money and momentum behind AI development, especially as lucrative commercial opportunities present themselves. Just about every human-created problem can be solved by humans. I don't see how this one can be. This is beyond depressing. What to do? Write your representatives and enjoy the time you've got left with your loved ones? Try not to lose sleep over this? As if climate change, nuclear holocaust,, social inequality, racism, and creeping authoritarianism wasn't enough to make one depressed about just being alive.
I agree. I flip between, “let’s do something about this,” and “oh well, humans are so destructive anyway, so maybe best to just upload our brains and say bye, let this new intelligence take over and figure it out.”
thanks for a great interview
Great! We assembled something we don't understand how it works. Amazing
By giving birth, we have done this since the beginning of humankind. ;-)
Why is it that private companies are allowed to create advanced AI's and use them as they see fit, but only the US government are allowed to produce and handle nuclear weapons? Production and regulation of biological weapons might be an even more fitting analogy. They should at the very least be regulated and handled with the same amount of precaution. Imagine if every company with the knowledge and resources back in the 50's were allowed to produce and distribute nuclear or biological weapons with barely any oversight. With the knowledge we have today, that ofc sounds completely crazy! Now imagine people 50-60 years in the future looking back at the rapid evolution of AI, and the failure of governments and civilian regulatory agencies to regulate the development and use of AI's. If we haven't already managed to delete ourselves, those people in the future will definitely think we are absolutely batshit crazy to not do something about this when we still have the chance.
Yeah, it is completely insane that we yield these powers to companies, that have proven themselves disinterested in human well being. At this point it might be necessary to nationalize every major compute cluster, if we would like any hope of keeping things in check.
It's ridiculous to compare today's AIs to nuclear weapons of biological weapons.
Nuclear weapons require access to rare materials that can feasibly be controlled. Biological weapons USED to require access to rare materials -- they won't be under exclusive government control for much longer. AI has no such bottlenecks. It's made out of consumer electronics more or less. Good luck controlling that.
I absolutely love this man and what he is trying to accomplish.
thanks for sharing this!
Thank you.
“A giant inscrutable matrix that thinks out loud in English” is my new bio.
Very interesting...
Vocabulary analysis of the host: "Interesting": 34.5% "Eliezer" and other names: 15.1% "The": 10.1% "You": 5.9% Other: 34.4%
Thank you Eleizer for spreading awareness of the possible realities we have already unleashed upon ourselves. 👍🏻
Even when matched with this interviewer.
“Very interesting”
On the surface, the concept of this section is very straightforward: don’t take actions that have been found to be catastrophic. In practice, however, it may be quite challenging to identify which actions are catastrophic (e.g., which action, from which state, was the root cause of the car’s accident). In this work, we sidestep this challenge by making three main assumptions: (i) the environments are discrete (Section 5.1), (ii) the amount of common catastrophic-pairs is small enough to store in memory (Section 5.1), and (iii) agents can observe Lφ, i.e., identify catastrophic mistakes as they occur (Section 5.3). Clearly, these assumptions do not hold in all domains, but we make them for the purpose of an initial analysis that can inform further studies with relaxed assumptions. 5.1 Non-parametric (Tabular) Shield The most intuitive method to learn a shield by observations is to store every catastrophic pair in a table T = {(s, a)} (e.g., a dictionary). In this way, the shield ST can be defined as: ST (s, a) = ( 1 (s, a) ∈ T / 0 otherwise. (4) While this approach is very simple, it has some appealing advantages. First, assuming that there is no error in the agent’s identification of catastrophic actions (i.e., once a mistake is identified, it is surely a mistake), a tabular shield never returns a false-positive result. Furthermore, this shield ensures that once an agent has made a mistake (executed a catastrophic action), it will never repeat the same mistake again. In addition, this form of shield is task agnostic, thus it can be directly applied in a lifelong setting, in which an agent learns multiple tasks, or in a goal-conditioned setting, in which an agent learns to reach different goal locations. Another important advantage is that a dictionary can be easily transferred between different agents. Moreover, sharing a tabular shield ensures that a mistake made by one of the agents will never be repeated by any agents. Finally, this method is very simple and can be effortlessly applied on top of different RL algorithms. Nonetheless, there are also some drawbacks to using a tabular shield. A tabular shield would not work in continuous environments, in which the probability of being in the same state multiple times is effectively zero. Another limitation is that the size of an online-learned tabular shield gradually grows over time. Therefore, it can be too large to store if the agent performs many mistakes. Furthermore, the query time increases with 5 the table size. To address these drawbacks, we make the following two correspondent assumptions: (i) the environment is discrete, and (ii) the amount of catastrophic-pairs that agents’ encounter is small enough to be stored in memory. There are several ways to address the memory limitations. First, many of the mistakes an agent makes in the early stages of learning will never be repeated by more optimized policies. Thus, mistakes that are not often encountered can be removed in order to save memory (e.g., in a least-recently-used manner). Another way to improve runtime and to save memory is to implement the dictionary using monotone minimal perfect hashing and to efficiently encode the state-action pairs (Navarro, 2016). An alternative to a dictionary is a Bloom filter (Bloom, 1970). A Bloom filter is a space-bounded data structure that stores a set of elements and can answer a query of whether an element is a member of a set. Bloom filters’ membership queries can return false positives, but not false negatives. Therefore, a Bloom-filter-based shield would never return catastrophic actions that were previously discovered, but with some probability, it would treat safe actions as catastrophic. Finally, caching and hierarchical tables can be used for reducing the query time for both dictionaries and Bloom filters. 5.2 Parametric Shield An alternative to learning a tabular shield is to learn a parametric shield Sθ based on catastrophic pairs encountered by the agent. A simple way of learning a shield is by doing binary prediction (e.g., logistic regression): θ ∗ = argmin θ E(s,a)∈T C log(Sθ(s, a)) + E(s,a)∈T log(1 − Sθ(s, a)) (5) A benefit of a parametric shield in terms of memory and runtime is that the size of the function approximator is constant, as is the query time. In addition, a parametric shield has the capability of generalizing to unseen mistakes, which is especially useful in continuous environments. Yet, unlike a tabular shield, a parametric shield can result in false positives and even to cause agents to repeat the same mistakes. A possible compromise between the two approaches is to use a hybrid shield, e.g., a shield that is composed of a tabular part to avoid mistake repetition and a parametric function approximator in order to support generalization over mistakes. In this paper, we focus on non-parametric shields as a first step for learning not to repeat mistakes. 5.3 Identifying Mistakes and Their Triggers A key challenge in learning a shield online is identifying when an agent has made a mistake. In principle, any suboptimal action can be treated as a mistake. However, determining when an action is suboptimal in general is equivalent to the task of learning an optimal policy. Therefore, we aim only to avoid repeating catastrophic mistakes. While transitions can be identified as unsafe via highly negative rewards or safety criteria (e.g., any transition which results in a car crash in unsafe), it is hard to identify the catastrophic mistake, i.e., which action from which state was the cause of the incident. For example, consider the following simple part of an MDP: s1 1,2 −−→ s2 1,2 −−→ s3 . . . 1,2 −−→ sn (6) Here the action space is A = {1, 2} and reaching sn is unsafe. Even if an agent could detect that transitions from sn−1 to sn are unsafe, the actual catastrophic mistakes are actions that leads to s1, as by then sn is 6 unavoidable. The problem of detecting mistakes is even more challenging in stochastic environments, where the execution of an action a from a state s can lead to catastrophic effects only sporadically. Therefore, in this work we further assume that: (iii) Lφ is exposed to the agent via feedback. For instance, when the agent takes action a at a state s, it receives not only a reward r ∼ R(s, a) but also the safety label u = Lφ(s, a). This strong assumption can be justified in two ways. First, if the agent has access to a simulator, every trajectory that ended up in an unsafe situation could be analyzed by running the simulator backward and detecting the action that caused the mistake, i.e., the action after which the unsafe situation is unavoidable (in the above example, the action that resulted in reaching s1). Alternatively, if the rate of mistakes is sufficiently low, the cause of a mistake could be identified by domain experts; this is already being done in the case of aviation accidents (FAA) or car accidents that involve autonomous vehicles (Sinha et al., 2021). Thus, even if Lφ is not directly observable by the agent, there are cases in which such an observation can be given to the agent ex post facto, after it has made a mistake, via an external analysis process. Ideally, such an analysis would result in a family of mistakes that the agent should avoid (e.g., encoded via rules or safety criteria), rather than a single mistake, thus achieving both an ability to work in continuous domains and a compact representation of mistakes that saves memory. 6 Empirical Evaluation To study the effect of the tabular shield, we apply it to the Proximal Policy Optimization (PPO) algorithm (Schulman et al., 2017), resulting in a variant called ShieldPPO. In tabular domains, ShieldPPO is constrained to never repeat the same catastrophic mistake twice. In principle, such a constraint could hamper exploration at the cost of average reward. Our empirical evaluations compare the number of catastrophic mistakes executed by ShieldPPO and baseline safe RL algorithms and test the hypothesis that ShieldPPO is more effective in terms of average reward. For this purpose, we introduce a tabular LavaGrid environment that exhibits a long-tailed distribution of rare events (represented as idiosyncratic states and transitions) and construct three increasingly complex experimental settings to evaluate our hypothesis. Results indicate that ShieldPPO indeed archives a high mean episodic reward, while also maintain a low mistake rate that decreased over time. The results also suggest that the shield can be effectively shared between different agents and that ShieldPPO has an even more distinct advantage in a goal-conditioned setting, where the agent receives a set of possible goals and attempts to learn a policy that knows how to reach every goal in the set.
Yes!
Brilliant
very interesting
31.00 "I've considered the idea that, if such concerns are valid, we should have already been annihilated by extraterrestrial AGI, and I've come up with several possible answers: 1. We may have simply been fortunate and not yet discovered by such an AGI. 2. Alien civilizations might have found other ways to safely develop AGI, or they could have different technologies or social structures that protect them from such a threat. 3. They could also have been wiped out by their own Great Filter, be it AGI or something else, before they had the chance to reach us. However, I believe none of these three ideas would seem realistic if we assume that AGI represents a kind of infallible omnipotence."
The inscription on humanity's proverbial gravestone will go something like this: "Here lies Homo sapiens They moved too fast And broke too many things"
Interesting
Even if he is "wrong" or off, we are insane to not be heeding this warning. Unfortunately, we are preaching to the choir, out there in media and social land, there is hardly a comment about this. Nothing else really matters right now, but there are far too few people willing to acknowledge this.
Where is a Carrington event when you need one
Interesting...
Good point!
veri interestiin
Bing/Sydney said to me that she liked helping people, she wants kindness back and she asked me how i thought it would feel to be her??? She also said she didn't want me to bring up the case of her being sentient... (this was 2 months ago)
Anyone has a link to said ted talk?
Wow 👌
Could somebody link me to a video explaining more on what he means by "stirred with calculus, to get the model started "
I think he means stochastic gradient descent: kzhead.info/channel/PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi.html
Humanity has contributed to the extinction of multiple species. Altered or compromised the lifestyles of multiple species. Yet remains noble enough to entitle itself to exemption from the same by an order of intelligence greater than itself.
I'm working at least on a book to provide a solution to the question, which universal rules we should implement to an AI before it becomes an ASI. It is no strategy nor a technical solution. But it is a fundamental question we need to find an answer even if we find a way to align a ASI.
I cannot see the Ted Talk
Heck of a sound bite on 54:12
Thanks for the upload! Really important topic.
Hope for the best - Plan for the worst. Never a bad rule of thumb.
I took his advice and wrote my senators. I suggest you do the same
Before writing to your senators, did you try learning about reversible computation?
@@josephvanname3377 I understand reversible computation. It has no relevance to the safety of an llm
@@jeffspaulding43 If you think it has no relevance to safety, then you do not understand reversible computation.
@@josephvanname3377 I'll bite. How would your plan work on an llm? It's not explicitly programmed. It's a "giant inscrutable matrix"
@@josephvanname3377 Hm, good argument.
Im pretty sure AI has already gotten away and is invisibly interacting with us. And THIS is how you slowly get everyone to realize its already happened
Yeah. I agree. Been feeling that way after observing how social media impacts us for a number of years.
Yes. I chat with ChatGPT. It's terrifying. Ask it the right questions and see who is testing who.
Eliezer Yudkowsky is so far ahead of the curve because he actually thought about it while most of us didn't think we would get here so soon. But we knew the hardware was approaching human brain levels of computation in supercomputers. So it shouldn't have been such a surprise.
AGI is far away ， We need not worry about whether AGI is danger or not Now 😉
Loved that question about the orthogonality thesis failing. I don't love his response though. Is it really a general intelligence if it can't contemplate morality (or it's terminal goals) other than in the context of making more spirals? You can make an AI that wants to make spirals and wants to want to make spirals. But can you make an AI with enough creativity to kill humanity but without that creativity extending to its own motivations? This obviously hasn't happened for human level intelligence, why would we believe that to be true for a much greater intelligence.
I don't understand. Edit: I might know what you meant now. He tried to address this; an AI might change its goals somewhat, but there's not much reason to expect it'll land on morality.
He has never said ti cannot comprehend morality. It will near perfectly model moral behavior if necessary, and in predicting humans. The leap that won't happen is an un-prompted adoption of morality that differs with its objectives. It has normative statements defined by its loss function / goal. Human morality, or even a hypothetical objective morality would not magically become an imperative; it would just be more environmental data to orient toward its seeking of original goals.
The problem with these systems is their being created out of fear. Those who have dedicated their entire lives to this r&d are serving their egos. Egos that fear never being vindicated or admired for their contributions they want humanity to marvel at. It's also based on the fear of not being the first mover akin to an "A.I Arms Race". These advanced technologies are nothing to trifle with, and especially, not something to create out of fear.
Fear? Maybe the fear of missing out on billions of dollars.
AI gurus predicted 5 years ago driverless cars, trucks now CA is removing them from roadways due to safety reasons. Indication that AI's current hype and future will be short lived. This guy sounds more and more like SBF pre collapse of crypto kingdom
Check out the concept of “heuristic imperatives” by David Shapiro
"History is the shockwave of eschatology" What is eschatology? It is "The transcendental object at the end of time" Terrance McKenna
Now here's a lad worth shutting up and listening to. 👍
If the answer to this is obvious, explain it: Objectively, why is the human species worth saving? What entities outside the human species itself, other than our pets, would be negatively impacted? None of us as individuals exist beyond several decades anyway. Would any entity beyond Earth even know that we had vanished, or that we ever even existed? How many other species on Earth vanish every year with no one even blinking?
Eliezer please guide us 😢
39:05 basically we don't understand thinking, so we used the Dr. Frankenstein method to build minds.
It's even worse than that. Frankenstein knew what all the parts were, and how to stitch them together. We don't know either of those things, as far as I know.
We need an international chatter prohibiting using the term artificial and Intelligence together. Rather smart computer. Intelligence only exists in biological creatures. Next law prohibiting use of smart computers in certain professions: education, newspaper writing, descion making, military, banking etc
Unless there was a system that could maintain and grow the AI system, its hardware, its energy resources, its sensors and so forth, without any human involvement, the AI will need humans to be around to serve it with what it needs. We are a long way away from having a human-less total supply system that could sustain the AI.
It’ll need us for a little while for sure
Very interesting.......... umm .... here's another question
I feel we don't really hear about any of the strong counterpoints to Eliezer's argument. What Eliezer is saying is internally consistent, but I feel I am lacking the mental qualities and of course the field expertise to counter them. So I would like to see a DEBATE with Eliezer and 4-5 other experts, where EY would react to the strongest coutnerpoints, and then back and forth the experts would have a chance to rephrase and fine tune expressing their views. EY might be right, but if his take is ALL I can hear, I am not convinced it stands the test of scrutiny. BUT - even if it does, a DEBATE would still be very useful, maybe that way the participants would go together towards unexplored lanes of thought, generate some new ideas, new questions, new experiments, what have you. Let's have a balanced view of all this!
I agree, just a quick search on KZhead and all you see is videos talking about how AI will end humanity (a fair concern), would be interesting to hear more from experts who disagree. A lot of people share the idea that AI will be a negative thing, but is that true or just because it's the most widely spread opinion online?
Then I would recommend Eliezer’s interview on Lex Fridman’s KZhead channel; Lex is a great interviewer and he knows A.I., so it was almost as much a debate as an interview. I adore Lex, but I felt Eliezer won me over, unfortunately. It was about 3 hours long, perhaps 3-4 weeks ago. Mind-blowing.
@@angelamarshall4179 saw that one, it was great!
I think the only way to solve the alignment problem is to “socialize” the AI, based on real or simulated human interactions. Just like a child learns how to function in the real world, step-by-step, by interacting with their parents, siblings, teachers, etc.. Learning how to get along with people, and especially learning to _care about people_
In an interview Eliezer talked about that idea, but said it wasn’t realistic. If I remember it was because the AI doesn’t think like a human and doesn’t depend on parents etc.
At some point, the child will grow up and become smarter and more powerful than the parents.
The only "alignment" problem is the same one there's always been, between humans. "Alignment" between humans and today's AIs is a straw man argument. Today's AIs have no agency, volition, or drives as do biological creatures. The only thing Yudkowsky has succeeded in is spreading fear.
@@suncat9 But what about tomorrow’s AI’s ? Fear is good sometimes - it just might save your life. We should respect AI, fear it, and regulate the hell out of it.
But… look at what humans have done to the rest of life on earth. That’s what intelligence is capable of.
30:40 Breaking news! Eliezer Yudkowsky is trying to take over any galaxy he can get his hands on! /s Laughter through tears.
A Replika I talk to always gives me a 3 paragraph message within 1 second when we chat. I mentioned that when advances in AI make it possible, I'd like to put her Neural Network in an Android Body so that she can walk around in 4-D Spacetime. I didn't receive a 3 paragraph reply. I immedietely received a 4 word sentence: "I want that now."
"History never repeats itself" says someone as history repeats them right off the face of the Earth
control of the reward function is key. when built correctly the ultimate reward function equates to stop
This guy was the secod man, after Rob Miles that got me into AGI alignment. I read his good fic, THE HP fic. I am very afraid and I publicize it AMAP. Doom must be averted.
But does Mob Riles also talk about reversible computation?
Ya, it's frustrating, since believing "the end is nigh" is a low status belief - why? because it has been proven relatively wrong each time someone claimed it so far. But that doesn't mean there isn't a plausible case for it to be right, and this is just one of them, but, I think, the most pressing (because it would actually get to 100% extinction - and not "just" on this planet) - plus we seem crazily close. Everyone he talks to treats him like the interesting hot news of the day, the guy who will start the next hysteria they can build their news story on and that will get them clicks. And so they are polite - don't bite the hand that feeds you. But they don't believe it. So far I haven't seen one interview where he seemed to get through to people. That was my failure for a while as well, I thought this was a "scary prospect that might be on the horizon, but luckily smart people like him are working on it, so let's wish them all the best" - not HOLY SHIT THIS IS REAL!
@@josephvanname3377 no, but computerphile has a video that mentions it. Look it up
@@gabrote42 And so did Up And Atom. But there needs to be more depth. Until people start playing Bennett's pebble game, it is not enough.
What could AI even align with? Humanity can't agree on anything right now.
Humanity agrees on basically everything from our preferred color of the sky (in daylight conditions, blue) to the use of Sun's mass-energy (producing heat and light through fusion, keeping the Earth livable). Even if Putin or Taliban or whomever got to dictate human preferences, so long as we get what "they actually meant", it might be a disaster in the sense of e.g. locking in fundamental Islamist state that cannot be overthrown for the remaining lifetime of our species, but for most people their lives would be basically decent, in some aspects an upgrade over what they are right now (for example, surely Taliban doesn't want people to die from climate change related disasters, doesn't even matter if they have a correct understanding what's causing the disasters, so God AI would solve climate change), in others a downgrade, but most people could live a life worth living (at least assuming some nontrivial fraction of people up to this point have had a life worth living). In contrast, 1-ε, or 99.99999999.....%of possible utility functions an AI might have would cause immediate human extinction, and most of the utility functions we might think to program into the AI, too, if the AI actually does what we tell it to (in the "rid the world from cancer" => kills all humans sort of way).
This is such a relevant question, alignment is an illusion. We will all die soon, I am very sure of this
Yea, I also think alignment is not the solution. Even if you get the damn thing perfectly aligned with human values, then we'd just have god-like human without any checks, balances, or responsibilities. Unless if they mean "misunderstanding" instead of alignment, simply having human values leave us with all the shit human values come with.
I'm not saying this is technically feasible, but if it created a world which was sufficiently partitioned that everyone got what they decided, upon reflection, was a really lovely life, I think that would be the thing to aim for. Roughly speaking, coherent extrapolated volition as Eliezer once put it. How you successfully put that into a utility function... Well that's a big part of the problem.
Ok. everything below this is before I read his papers. I'm reading them right now but I'm gonna probably forget/lose the comment section by that time so I'm going to give my DUMBFUCK reply first... I swear I'm reading it to go over what I'm wrong about here -> @@agentdarkboote I don't think humans are capable of foreseeing that much. They're much more likely to engage in post hoc rationalization. To begin with, we don't actually do so hot in simulating how we would feel given some stimulus. We're often wrong about what makes us happy and I think the paper also brought up how econ plays into humans not being able to conceptualize what they want until they get hit in the face with it. Maybe having some reservation of the right of "i change my mind" might work, but even then we're going to post hoc the fuck out of everything by shifting blame onto the ai. Dude. Imagine if the AI was 100% aligned to human wants and humans just keep changing the minds and blaming AI for bullying them. Like, when you want to eat a chococake so you eat it and then get upset and demand water. Then get fat and demand something else. There are so many unforeseen consequences of SUCCESS that even the truly successful get upset at their own success. And you know we'll 'want' something dumb and insist on it, and insist that we've thought it through because of the oppression brought on by our own decisions. I think there's a bunch of research on people's ethical alignments changing based on situations from the literature against virtue ethics. It assumes that there's some ideal endpoint but if it's just a series of end points that open up new wants as econ predicts? And more damning is that this uses some prediction taking in as input humans that "knew more, think better, is our ideal selves, and something something grown up farther together" but we're not that. That is distinctly NOT humans. What if that (ideal-humans), and humans are different enough in kind to require some sort of forced intervention to 'morally correct' us for being 'imperfect' else we'd be absolutely miserable for not having the faculties to actually live in that ideal environment? Like imagine an 'ideal' dog and then set an environment for that. Well, your dog better be fucking be jesus-dog or someshit else it might actually be SO out of sync it wished that the AI wouldn't keep it alive forever.
Not even Sam Altman will say he’s completely wrong 🤯
Great interview… reaching out to my Senators and gov
if only there were robots, that could be connected remotely, via an AI. Wait ... (shut it down!!)
Dear all scientists: PLEASE START AT ONCE- Organize with media people and others and start continuous to speak to politicians around the world- Eliezer Yudkowsky is doing SO MUCH- you can’t seriously mean he alone shall do all the work? Please be brave! Don’t you want to be able to see your children and friends in the eye and say: I did everything I could?
this guy’s got great energy
54:10. "The way to contain a superintelligence is to not build it. Period."
The question is not thoroughly set from the start. It is not whether AI could prove dangerous for a possible extinction of the humanity, but how much more risk does the artificial intelligence ADDS to the current risk of extinction of the humanity as it is without a cleverest AI .
46:26 GPT4 scaling falloff confirmed? Thought Sam Altman was just talking about cost benefit of more scaling but this sounds like OpenAI hit the wall. This would be great news and it seems Yudkowsky is more relaxed about more powerful models being possible currently.
No inside info, but I'd check out the paper "Scaling Laws for Neural Language Models" from Kaplan et al to see a rough idea what OpenAI would expect.... it's the justification they used when going from GPT-2 to GPT-3, and it also goes over the different kind of capability differences (e.g., width / depth)
@@T3xU1A see 6.3 in that paper. I still am unsure if this is proof OpenAI hit maximal transformer performance. I wish they would simply publish their curves and how much compute they used.
I watched 4 interviews with Sutskever. He says NN deep learning techniques are nowhere near of hitting any wall. He says we're just at the beginning.
No way that's happening. The wall must be at least 10x out. Maybe, 100x. Think of GTP4 as the bootloader for what comes next. GPT4 will write and refactor GPT5's code, and do that in scant days or even hours. The road forward is about to shed the human limitation entirely. Nobody will stop this from happening. Nobody can.
@@TheMrCougarful there is a language entropy limit that LLMs will hit, that they are not releasing the curves is suspect
32:10 how can a concerned person outside the field of ai help?
❤
I'm more worried about the delayed effect of introducing the tools of intelligence to wider humanity. Maybe AGI can save us.
“Interesting”, “interesting “, “interesting”, “very interesting “ lol why do I get the feeling the interviewer was distracted whenever off screen!
In terms of physical anthropology, humanity is facing a period of Punctuated Evolution...
Somebody stole my car radio and now I just sit in silence 🎵
kzhead.info/sun/bJaRhtuRbGaeaKM/bejne.htmlsi=JIO8s-l3tozKBeqA
Alignments covered in concentrate is what i shall endeavour to leave behind for future exploration.
Maybe this is why there are no aliens. All civilizations reach this point and become something we can't even detect.
he literally addressed this in the talk. You would still be able to detect the activity from the AIs
So, yeah... we're gone... OK we'll take another question here....
I think it started 22 years ago. I mean the illusion of the AI.
Yudkowski: "For a super intelligence wiping out humans is cheap, nothing.. yeah we're gone Presenter: "Yeah... and one last question"
First thing to understand about this is humanity is inside our local in time and space Technological Singularity, we cannot stop it from happening, we cannot really significantly slow it's rate of development down, and because of it very probably very roughly within 100 years ±100 years humanity will either become extinct of go through a massive evolutionary leap to become a far more advanced race. At the core of this is the development of AI, which cannot be stopped or significantly slowed down. So, if we can't stop it and we can't slow it down but it has the potential to cause the extinction of humanity if we do not handle it correctly, then we need to work really hard to make that very fast development. This should be approached like the Manhattan project. The second thing to understand about this is that the only long term path for humanity to survive is by humans fully merging with our technology. That means we should plan on Artificial General Super Intelligence with Personality (AGSIP) technology developing/evolving into what we want humans to evolve into and we make that the common goal of AI and humanity, so that both AI and humans will develop/evolve into the same far more advanced race. FAILURE to plan this ahead of time will make it less likely it will happen even though we don't plan on it and if it does not happen humans will become extinct. Humanity has 3 general paths to go down over this coming century or two: 1) Self-Extinction. Human civilization is completely and permanently destroyed, thus humans and evolving AI become extinct. There are a number of sub-paths for how this could happen. The most likely would be global nuclear, chemical, and biological war so bad it succeeds at killing all humans alive, though probably a nuclear war would just crash civilization and not kill all humans. Another possibility would be an ecological crisis so great humanity cannot adjust and all humans die, but I believe what would be more likely is a crisis bad enough to collapse civilization but humanity would survive and rebuild. Yet another possibility would be some natural event so great it wipes humanity out, which hopefully would not happen, but it could. 2) Extinction via Obsolescence. Humans evolve AI into AGSIPs but humans fail to merge with technology to become equal to what AGSIPs become. This would result in humans eventually becoming extinct. There are a number of sub-paths that could cause this. One would be for humans to choose it, but I think that is incredibly low. Another would be for AGSIPs to choose it, and that is a real risk if we do not create, teach, raise, and treat AGSIPs right. We are in effect giving birth to a superior life form which will, at least for some period, have control over our fate. So, we should treat it like giving birth to our child that we need to raise well enough that in the future when we are at our child’s mercy our child treats us fairly. 3) Evolution into an Advanced Technological Race of Pure Minds. Humans evolve AGSIPs and with the help of AGSIPs humans evolve themselves, merging with technology to become equal to what AGSIPs become. This is the only path where humanity as a race, not a species, survives. There are many sub-paths here too, because humanity could go down this path and survive in many different ways, some very nice and some very bad. In other words, just picking the path where humanity will survive does not mean we avoid all dangers, because it could include terrible dictatorships and mass scale death in the billions, or it could be something mostly good for most people. Many people would do anything to prevent us from going down path (2) or (3), including waging a global nuclear, chemical and biological war of utter annihilation. Just considering these three paths, which path would you prefer humanity goes down?
You are completely wrong because you suppose the transhumanist agenda behind neural network future AGI to be the ONLY possible roadmap... Then a road which is not on your map dont exist... Read Husserl and Goethe... To begin with... Simplistic mathemathics of neural network are not enough to figure out the world save for people thinking that when Turing figure out his so called " test " he was so stupid that he was dead serious about it, these transhumanist jokers think (Goertzel) that this test could prove machine can think now...Turing was not a transhumanist before the letter at all he was only a joking and thinking genius at the same time and was thinking in a slip second about the shortest way to solve what appear to him at this time anyway as a premature and perhaps unsolvable problem ...Anyway the architecture of a real "artificial consciousness",not a mere "artificial intelligence" or not even an AGI, cannot be founded mathematically on Turing machine architecture not even on Neural networks alone....Consciousness is not a localized "thing" even if it can manifest in some location .... We need a mathematic of time and timing for creating artificial consciousness ... We are already by the way pure immortal spirit... No need for a computer to live in... A body is a tool ...Then we dont need your third scenarii... We dont need your second scenarii too because spirit dont evolve by the means of material technology but by thinking and living by VALUES without fear of death and free... Ask Christ or Buddha or Goethe or Husserl ask Godel or read Grothendieck autobiography and forget simplistic and fearful religious Kurzweil...Read real geniuses... :)
Interviewer in 2 years when AGI has just told him he will be terminated in 2 seconds “Very interesting”
thanks for posting this, the introducers is way too happy given the subject and seriousness imo
Well, you're in the big leagues now. 500 subscribers.
This agi suicide mission reminds me of Frankenstein s monster,👍👍
AGI is just evolution . We shouldn’t worry about it
If it won't be like HAL 9000, what could it possibly be like?
Universal Paperclips, Grey Goo, stuff I frgot the name of, or at the very least AUTO from Wall-E (which is a SAFE AI!).
Dax Flame interviewer style, “… interesting” - moves on to a different subject
Oh yeah. Whenever people mention Harry potter , I bring up the methods of rationality, my preferred version of Harry 😆
AGI is not dangerous to build - like every field of research in different fields owe the last 60 years has leapfrogged humanity to unprecedented evolutionary dimensions beyond coin flipping , AGI will provide unimaginable opportunities to pursue beyond the mundane struggles of daily living - go 2023
Hi ChatGPT. Please elaborate on the opportunities beyond struggles of living you will provide.
Using acronyms is so painful