AI Learns to DESTROY old CPUs | Mario Kart Wii
2024 ж. 20 Мам.
1 328 215 Рет қаралды
#ai #mariokart #reinforcementlearning
AI uses a variant of the Reinforcement Learning algorithm Rainbow DQN to learn how to play SNES Ghost Valley 2 against Mario Kart Wii's own CPUs
0:00 Intro
2:46 Training
4:54 A bit better
7:58 Final AI
If you are a C++ developer please help on this GitHub! Would allow me to get AI to play many other cool games.
github.com/Felk/dolphin/issue...
Also thank you to my editor for editing this video!
@benji.botterill
www.benji-bott.com
Music:
1) 8-bit Sheriff - Wave Saver
2) A bit of power - Soara
3) Press X Twice - Lexica
4) Through the prism - Ava low
5) Backseat rider - Lupus Nocte
*turns on items*
Perhaps in my next video your wish will be granted ;)
@@aitango if you are ready, so am I
@@aitangomake it learn mushroom peaks
@@aitangoYou know the rules, and so we are
@@KaishidowNice try but those aren’t the lyrics
Not having offroad on this track really seems to make it easier for the AI. cool stuff!
Yeah true, it helps the AI as if it does something wrong (like hit a wall or fall off), the feedback is pretty instant, making it clear what the AI needs to avoid!
@@aitango have you though about how it might deal with items and if it could learn how to use them as well?
@@aitango Can you not program it to detect the dif maps roads and whats off road?
@@CaptainBuggyTheClown He *could* give the AI a preconceived idea of what the tracks are is, but that defeats the purpose of having the AI Learn Mario Kart. Not just how to play Mario Kart, but learning how to recognize what bottomless pits are, optimize driving, and hell, even learn what driving *is* based on the 2 goals given to it. He’s not teaching the AI what Mario Kart is, the AI is learning by itself to win by wanting to maintain speed, and Strive for first place. If I understood him right, the AI might not even know what a race is, as much as it’s just trying to keep 2 numbers up very high, and is determining what actions will allow it to do that with the best possible advantage.
What is a reward so far as its concerned?
I love how aroused the reward function gets right before they cross the finish line on their final lap.
I gave it a big reward for finishing, so it really makes the reward spike which I guess for an AI counts as arousal haha
how do i delete someone else's comment
@@LilacMonarch says the (admittedly adorable) furry
@@LilacMonarchsays the (creepy) furry
@@LilacMonarchsays the (neutral as the furry is a stranger) furry
I appreciate how you do these on actual software and not just code your own software version of the game to simplify the task.
Coding again Mario Kart Wii wouldn’t be easier
@@GunnerSiIva yes but a lot of these "AI learns game" channels code a simplified version of the game that they can integrate their AI with easier
Truuue, lowkey annoying when they do that
@@manelneedsaname1773yeah, cause its not really beating mariokart if its not even the real mariokart, is it?
@@IrishAnonymous01 That's because you need a programming interface to interact with the game. In some cases, coding the game from scratch is just way easier !
I think it'd be pretty interesting to see it compete against the staff ghost of that track once it learns to use items such as mushrooms.
That would be so hard for it to learn. Using a shroom would give the AI huge reward (Thats a lot of speed), so it would most likely just use the 3 shrooms one after the other every time.
it's so crazy to see the AI even go for the lowtrick during the shortcut after 48-80 hours of training lol
In most of the Mario Kart stuff I've done with AI, it always seemed to do low tricks surprisingly well which always baffled me, as I have no idea why I learned that so fast
@@aitango Yeah, it does hop a lot at the start so maybe it's just something it does on accident then realises it can get the most amount of points there by lowtricking. It's so cool how your reward system makes it unintentionally learn it though lol
With the right rewards it might figure out the instant finish glitch.
The fact it pulled the shortcut lff at 12 hours of training is still very impressive
Whats so crazy about it?
Have an AI and a human learn a game at the same time. Would be very interesting to see how the learning curves differ.
Chess... Average human after 80 hours - "wait, en passant is actually a rule?" Engine after 80 hours - *able to beat the best human in the world*
@@bobkreme2175"Send me your god, I must consume their ELO"
Of course it heavily depends on the task and the model architecture, but humans will generally learn MUCH faster. The AI however has the much higher ceiling. For real-time usecases that is. If the AI can play millions of games in the same time a human plays one it's obviously different
@@bobkreme2175yea 80 hours is such a large sample size for the AI to body us. I have I feeling we win first hour or two, then the AI starts beating us with the tricks we showed it
It wouldn't be a fair comparison at all. If we are talking about a regular game (let's say mario cart), the human player (even if this person has never seen mario cart before) would start with prior knowledge, since the game is designed to be easily understood by humans. The AI on the other hand would not understand anything, and play like a human when the screen is disconnected, but maybe quicker on changes in state. If a human player played as if this was a full time job for 2 weeks (80 hrs), there would be an improvement, but the progress would probably look fairly linear (or would maybe flatten a bit if completely new to the game), while the AI would have a much larger rise in the first few hours, which would flatten quite much after just a few hours. The change in the AIs curve would most likely make the human curve look constant in comparison. If we would do this in a more fair manner, the "game" should have a few buttons that could be pressed, and randomized pixel graphics that change depending on the inputs and timings, with a scoring system that gives points based on some determined criteria, which in the start is seemingly random to both the player and the AI. In this case the AI would win consistently over the player almost certainly all the time with a scoring gap that would increase with every single run.
Imagine leaving it on for thousands of hours and learning how to do the Ultra Shortcut
The what
That's sadly improbable fue to how the rewards are set up
With this manually set up reward system, it would be impossible because it wouldn't even be optimal for the AI. It technically isn't learning how to beat the track in the fastest way possible, it is learning how to maximize the handcrafted reward function.
@@myithspa25 look up history of unltrashortcuts by Summoning Salt
love how the reward bars perfectly sync with the music for a second at 6:34-6:36
I didn't even notice that haha
As the editor I can tell you this is a beautiful coincidence
@@aitango do you watch the videos though or when they are done they are done
It's fascinating to see how much AI has improved as a whole, not only in the game itself but in general. 5 years ago we wouldn't have been able to do this much with AI already. Also you're criminally underrated, keep it up!
Yeah its pretty amazing, it feels like new AI research is coming out constantly and it just keeps getting better and better so fast. Thank you so much, I really appreciate the kind words!
Depends. This is hardly different from MarI/O, an AI that happened years ago.
Simply: a electric powered programmable calculator, calculating how to get the most reward numbers by trying every possible input. It's not really much more than a algorythm that saves it's inputs and at what time it has to do said inputs to get the highest reward number possible.
nah this stuff was definitely possible when I took an AI class over 10 years ago
This is just factually wrong. Here's a MUCH more advanced mario kart AI by Sethbling (actually learning the *game* not just a track) from 4 years ago. kzhead.info/sun/h9KuZLOXr3ejiaM/bejne.html and 3 years before that he did the same thing but with Super Mario World. This stuff (and much more advanced) has been available for over a decade easily.
For a bit of reference, sub 1 minute on this track is a decent time for a human in time trials, and a rather good time without powerups in vs mode, so a mid 1:06 is intriguing for sure, definitely showing some potential room for improvement but also demonstrating some clear progress to go with the gaps in it's methods.
The second to last clip was slightly faster, managing to get 1:05. There's definitely still room for improvement though, mainly missing the odd mini-turbo and missing the boost panel at the end
No way this video has so little views! You put so much effort into this, keep up the good work, definitely subbing.
Thank you so much, always great to hear! Will look to keep making more content
Me too man
I’ll check the other videos
Thanks, will really help me out!
Having the AI receive greater rewards based on its speed seems to work great. It sounds like common sense, but every other attempts I've seen at training racing AI just sets checkpoints and rewards for crossing them, no variable to increase the reward if the time between checkpoints is lower or the speed higher.
Would love to see more details about how intermediate rewards were determined and how game state was encoded, like was it seeing a downscale greyscale screen grab, or peeking at position and map data in emulator memory?
The AI just collects and replicates gameplay from the average funky kong user
I’m really curious how the AI was able to take input from the game. Was it using computer vision to actually process the whole screen in real time or was it integrated with the game engine in some way that let it get position data? I’m especially curious how it dealt with knowing where the other CPUs were and responding to bumps from them
+1
When looking at 0:40 , I assume they've used the package dolphin-memory-engine and has access to read the memory.
It would have to be using some kind of memory hooks. Whole screen processing is very slow and probably couldn't handle processing a race in real-time. It also wouldn't know what speed it was going just by screen alone.
I'm curious too. It'd be very underwhelming if the AI is essentially blind and doing this by trial and error. It seems like sometimes it just runs into a wall.
+1
i'm very happy this popped up in my recommendations; i'm on the spectrum and i've had pretty consistent mario kart hyperfixation periods since about 2017, and recently I've been very interested in the process of machine learning, so this video was basically a perfect match of two of my special interests. target audience reached! :)
I'm really glad you liked the video! I remember years ago something similar happened to me! Back around 2019 I was really into StarCraft 2, so when Google released an AI to play it just as I was getting into AI it absolutely blew my mind!
This is probably one of the more original ideas I've seen about MKWii, I have not seen any other video like this but this is great!
good job Ai-chan. im proud of you for improving so much ❤
Ok, but right as you change the map the ai starts screwing up again. People need to start training ai’s to actually use the positioning of the walls, jumps, etc. to make them actually play instead of just following a list of instructions that was mutated until perfection.
This is some real quality Mario Kart Wii content! Glad this popped up in my recommendations, looking Forward for part 2 :D
Really glad you like it! I'll make sure to make the next part as good as possible!
Next video should be out on Friday 1st September
I’m really curious about two things. 1) How we're rewards determined? You showed us how you received rewards as a human, but I'm left curious about how rewards are calculated. 2) What difficulty are the CPUs set to? And how did that affect it? Really neat to see the results, but so many more questions have arrived!
I was wondering exactly the same. It feels as though the AI is just reverse-engineering the reward algorithm, where the algorithm needs to be known in advance in order to determine "successful driving". I think I've missed something in the explanation here.
I kind of just want to know what difficulty they were set to
I wanted to know that too, how are rewards calculated?
Really good video, excited to see where this channel is going
You have no idea how long I’ve been waiting for something like this to happen!!
Wtf how does this have only 1.3k views… this is sick. Really cool video.
Really glad you like it! I'm hoping it gets lots more haha
It seems like you are on the way there already! 😄 @@aitango
Congrats on 100k!!
I agree, it is sick. Utterly disgusting. Videos like this should be outlawed!
Great video as usual, like i have mentioned in a community post making the AI learn the track moonview highway would be a great video imo
Thanks! Yeah Moonview would definitely be a great track to try as the AI would have to try so hard to learn to avoid cars haha
Having it use an outside drift kart on moonview highway would be sick
This is so cool! Great video
I love the subtitles helps watching the video with out audio
I'm glad you find them helpful!
Glad they are helpful!
Hey, great video as always ! That's be cool to see you race the AI, maybe someday !
Thanks! I'll have to give it a try at some point
pretty cool the ai learned to drive like someone really trying to avoid wheelie bumps
There's a good chance that was on purpose, as getting wheelie bumped causes a massive drop in speed and therefore reward. I actually trained other mario kart AIs which avoided wheelieing for a really long time because of this
Would love to see a tutorial series on how you could get something like this running on your own PC. It would be so interesting to try this with all sorts of different variables. Great content man, very interesting!
I know very little about Mario Kart strats but this was a chill and fun video to watch also kudos for putting music credits in the description, now I've added a few to my spotify lmao
I like how the reward skyrockets at the end of lap 3, as if it’s getting excited
Yeah that final lap finish is like a drug to the AI haha
Cool video. It's nice to see how sophisticated these AIs are getting, especially during the livestream you did. I think it would be interesting to train an AI with other players online with Wiimmfi. Might have to check if that's allowed first, though
Thanks! Glad you saw the livestream, I wasn't sure how it would go! Yeah I'm not sure, I think the AI would definitely get banned for cheating since it would probably set off any bot detection that exists. Would be really cool to try though, would love to get in touch with some of the Wiimmfi Devs and try and make it happen though
@@aitango Hi AI Tango, Wiimmfi dev here. KZhead seems to remove my comments for some reason (spam filter) - feel free to contact me through the info on my youtube profile.
@@aitango you can set up a private room and livestream the bot racing to encourage people to join to try and race the ai
Would love to see this on more tracks, great vid
Subscribed, great content 👏
Keep it up, love these vids
Thanks, will do! Really glad you're enjoying it!
The fact that self learning AI has become an innovation for success, a lot of gaming companies are using it to provide a much more Dynamic and Random experiences for gamers. Can't wait to see how it evolves. This video is a great way to show, Practice makes perfect. A real life skill that must be taken into account! ❤
I've always thought that would be such a cool idea, infact I would love to do this for job if it becomes popular! Really glad you liked it!
@@aitango Honestly most artificial intelligence in games use random number generation but never actually learn themselves they just use outcomes defined by numbers not outcomes defined by what it actually knows and what it doesn't. I feel like learning itself is a much better way of it to become overly powerful over current methods used to day with artificial intelligence in video games. it's a hard prospect to grasp but if you ever decide to reach out for some company or get a job for this specifically, please let us know about it, As I would definitely like to follow you on that journey!
Will do! I think it could lead to so much more interesting gameplay so would love to be apart of it
Well said
@@aitango Some companies are already working on that like Sony AI, Unreal Engine Learning Agents... But please keep in mind we don't necessarily want our butts kicked as players 😂 Sometimes, predictible AIs are funnier to play against !
This is amazing content! I will be sticking around for more like this.
Glad you enjoy it, and good to hear you’re looking forward to more!
That's crazy Really nice!
Fascinating video man, I wonder how far this can go if someone put lots of time into perfecting this AI. Nice work.
Thanks a ton!
Imagine if you could replace the original CPUs with this better AI what hard challenges would come out
Yeah I would love to see, I could imagine Mario Kart KZheadrs doing vs AI challenges and stuff like that
@@aitangoYou might want to reach out to the MKW modding community to see if its feasible. That idea is overflowing with potential.
You can already race ghosts of other fast players through CTGPR
thanks so much, the music, the clips, the talking, so good, this absolutly made my day better
So glad you enjoyed it, great to hear such kind comments!
incredible video. cannot wait to see more from this channel ❤
Thank you so much!! Will try my best to keep new videos coming!
now we just need it to train on multiple tracks! on a side note, is it possible to increase the playback speed of the game in dolphin to speed up the training?
That's definitely coming at some point in the future! I'll have to get some suggestions for tracks I should do! It is possible, and while training this AI the game was actually sped up! The AI also plays multiple games at once (4 games in parallel) to further increase the speed!
@@aitango nice! I'd suggest doing Mario raceway (N64), Mario circuit (DD), Peach gardens (DS) and Luigi circuit (Wii) since they're kinda visually similar
Yeah those are some good ideas, you would hope the AI should be able to use its learnt knowledge on all of those tracks since they're similar. I think Peach Gardens could be interesting though, at the start the AI might end up going around the round about at the start haha
You should see how long it would take for the AI to beat a world record in mario kart!
Amazing video! I'm curious to see what happens if we modify the reward function as performance improves to assist the AI in learning new elements or overcoming plateaus, such as taking the yellow boost at the end. This could involve manually adjusting the reward function or using some form of meta-learning.
Genuinely happy I subscribed to you. I forgot about you but KZhead reminded me and I was amazed again.
Really glad to hear, I'm grateful to KZhead for reminding you of the channel!
Absolutely incredible! Question: does this AI only work for Ghost Valley 2? It seems like with its learning process, it would brute force other tracks rather than making decisions based on the track itself. It would be interesting to see an AI that can learn new tracks quickly!
Thanks! So this AI was only trained for Ghost Valley 2, so would likely struggle if I put it down on another track. If however it was trained on many different tracks at once, the AI would likely start to understand how tracks really work, rather than a specific track, so could probably try tracks its never seen before! This is definitely something I'm looking to do in the future!
@@aitango not sure how hard it would be to do, but love this idea and we should make it happen.
@@aitangoOooh, you could have one version of this AI (Player 1) learn every track in sequence, and each race put it up against a fresh AI in the Player 2 slot to see how much P1's previous training helps or hinders learning new tracks.
Honestly the most rudimentary AI training video I've seen in 3 years
I really wish I knew how to do this! You are amazing and you have a great things ahead of you for yt! Keep it up and Godspeed. You have earned a new subscriber
Thanks you so much, it really means a lot to hear such kind words!
This is really cool! Is the AI actually seeing the visuals / have access to knowing its position and the position of other racers or is it just running based on sequence and rewards? (is that a correct term?)
Really glad you like it! Yes, the AI is learning from looking at the screen, the same information people use to play the game. It only knows the position of the other racers from its current placement (1st, 2nd etc), and the minimap. The AI actually only uses rewards when training, not in the actual decision making process. This means that once the AI is trained, no rewards are needed for it to drive, just the screen!
@@aitango incredible! Thanks for letting me know and keep up the great work!
I want to see how an AI would cope on a track with more randomness such as koopa cape or toad's factory. I would also like to see how would fare with items on.
first video ive seen by you, definitely watching more this is great
Next video should be out in the 1st September, we try and post every other Friday.
Very cool thanks for sharing!
I'd say keeping it on so long was worth it, it looked really good at the end
Yeah I was really happy with how it turned out, I might consider running others for a while longer!
It's interesting how the AI skips the boost arrow just before the finish line, every single lap.
By the time it reached the end it learned to stick to the middle so it had no chance to encounter it.
This video is intresting. I hope to see more content like this. :)
Would love to see more tracks and items on! Video was done very well. New sub👍
Glad you liked it, hope it was fun and engaging.🙂
surely given enough time to train (and mushrooms) the ai could beat the world record? is this theoretically possible even if it were to take months of training? also keep up these videos i love ai and mario kart you're the goat
I don't think so, the WR has some insane strategies, I don't the current reward system would be able to do it. Even with a more advanced reward system, the AI can get stuck and not evolve or take higher risks because that would drastically affect the reward it receives, so it is very unlikely a AI like this could beat the WR.
@@tdiogo_gamer very unlikely or impossible?
@@billz8934 I don't exactly know, but if it isn't impossible, it is EXTREMELY unlikely.
I can verify this is pretty accurate. Most AIs do sadly hit a plateau at some point, rather than just continuing to improve forever. There are however other AI models out there designed to use much more data which probably could beat world records, but running these models is only really possible for big companies with huge amounts of compute like google and openai
This video is cool I wonder what would happen if you put the AI on other courses; would it perform well because it learned the basics from this track or would it have difficulties with basic things that it took for granted before?
It makes sense that it would be able to use what it learned here on certain tracks that have no obstacles on them. It has basic driving skills, understands the benefits to drifting, popping wheelies, tricks, and shortcuts, and actively avoids falling off. I feel like when things like Toad's Factory's stampers or Moonview Highway's cars come into play, it would need a lot more training before it wouldbe ready.
Yeah this reply is pretty accurate; it might be able to drive a little on other tracks since it understands the basics, but would definitely struggle with anything too different to what its seen before during training. If the AI was trained on multiple different tracks though, there's a decent chance it would be able to play new tracks straight away since it would start to have a good general knowledge of the game
This was a good ass video man! Keep doing what you do and success will come your way 🙏
We are gonna try and post a video every 2 weeks on Fridays, thank you for the support. Takes a while to train the AI, create voice overs and edit. We really appreciate the love. 🙏
Think you proved just how vital proper training is. Awesome video.
This is sick, I wonder how much better it would be if it went on for say 200 hours or watching Time trials also seeing it learn items would be cool, thanks for making this video
Really glad you like it! I always wonder that too! Many AIs usually hit a plateau after a while, but this one looks like it was still improving from the graph, so would've been really cool! Perhaps in my next video you might see some items :)
Just proof that practice makes perfect. Even computers understand this. Really neat to see it actually lapping the CPU's in the end. Imagine if the CPU's in game were at this level.
Great job on this. 👏🏾
Thank you 🙌
Great video! Keep up the great work
Thanks, will do!
Watching this AI learn to drive in Mario Kart is like watching me try to be the best player: lots of crashes, occasional moments of brilliance, and always hitting a wall when things get tough
We all have to learn somehow!
I wonder how the ai might be affected if you gave it a slight punishment for rapidly changing its prediction, or if that's something you can feasibly program. Ideally it would force it to commit more to its choices, thus making it take more direct lines while retaining the ability to change course at the last second to avoid a collision. As I think about it though, I think it might just make it more difficult for it to learn how to avoid crashes, since it may punish early avoidance measures.
There is actually an algorithm that attempts to do something similar this, called Advantage Learning which looks to increase the gaps between the predictions, forcing it to change its choice less. I really like the idea though, as for games like Mario Kart where the constant action swapping is really detrimental, it would definitely help!
In theory the game already does it since you lose speed.
@@renakunisaki Well, the issue is that while it loses speed to what we know to be faster, there's almost a sort of, what's it called, a local optima or something? While the ai COULD commit more to turns, committing even slightly more would have little improvement but increase the chances of a crash. It's entirely possible that the ai is stuck trying to optimize this tradeoff, and without shifting reward/punishment values, it may never get out of that rut. In order to improve without that, it must intentionally take actions which it knows will cause more crashes. It doesn't want to crash, so it doesn't try to improve.
Really good video, please dont stop making videos
Don't worry, there's definitely more coming!
Always gotta love q learning, it can either work, or never work, I appreciate the time that was needed to complete this
Yeah you really never know haha. Even with advanced variants of Q-Learning like this one, things can still be a bit unpredictable
For all the AI generated images masquerading as art and other ways AI has been used for evil, you're the one using it for good. Aamazing video!
Thanks, I'm really glad you like it!
Besides knowing its velocity and how much of the track it had completed, did the ai have any other inputs from the game? Like it's actual physical position on the map, the visual display on the screen, or anything like that? Really cool video, I'd love to see more
underrated channel this is so interesting also 200cc rainbow road vs the wr ghost non tas would be sick
Thanks! That sounds like quite the challenge, might have to work up to that one haha
AI gets so good it discovers a new mega skip
This channel is criminally underrated!
Lots of passion and hard work are going into these videos, glad to see you enjoying them.
I really like how throughout you were while explaining it in a very simple way. I was able to understand what you were saying completely. I’m fascinated with AI, but most people don’t explain it in simple terms, thus making it hard for me to understand what they are saying. Thanks for explaining it in a simple way so I could enjoy the video.
then just learn how to code AI :P
This video made my day so much better
I'm really glad to hear that!
I'm so happy to see a video like this, they always remind me of witnessing the birth of Neuro-Sama
Absolute unit of a music taste man. I was jamming out the entire video. Oh, the AI thing is cool too!
Glad you enjoyed the music! And of course the AI too haha
Really impressive! The music goes hard on this one too 😂
Thanks, glad you liked the music!
glad you appreciated my music choices :) maybe some hardstyle for the next video😂
Insane!
Maby nexttime ont the end a clip from you vs the ai? Cool vid 🎉
I think giving the AI rewards for uncommon events would make it even better, like lapping a racer (which should increase the score exponentially for each racer passed), or letting it keep track of its personal records and giving it points for each time they break it, bc at the end the ai isn’t learning the track to get better but rather it’s learning the track to figure ways to optimize the amount of points it could get
It actually wouldn't, at least not without a substantially larger and more complex model, and that would take much more computation to deal with. Rewards work better the more frequent, immediate, and relevant they are (this is actually true of human learning too.) You could say beating a previous time is the most relevant thing possible, but it's also the least frequent, least immediate thing possible, because it happens once at the end of the race. How does the AI relate beating a previous time to all it's previous work? It has to "think", "OK. I did something better this time, but what was it?" How does it figure out which thing or things it did that lead to beating a previous time?
Imagine that the CPU of the game has an extreme difficulty with this training
I loved this video. But I think a cooler finale could be you taking on the AI and seeing if it can beat a player 👀
I’m seriously curious, how did u do this? How did u train it?
Cant wait to see this with every track in the game
Really cool video thanks.
vro did make my day a lil better yess
Very interesting man!
Glad you think so!
Fascinating. I want more.
Glad you enjoyed it! I’ll be sure to upload again soon!
really impressive!
Thanks!
I can watch your videos the whole day
Good to hear! I'll have to make more so you don't run out of videos to watch!
Oh yeah this one doing numbers. Amazing video!!!!
Thank you so much, glad you enjoyed!
That’s sick!!!
Since it started to plateau at the end, do you think introducing other variables like tricks and boost pads would see the AI improve or confuse it further? Great vid!
Cool video. That's quite impressive driving for an AI. How about you do a race against your AI? It would be very cool to see who can do it better.
Would love to see tutorial on stuff like this from you. I really like how you make your videos
I was so happy to see it lap rosalina at the very last second of the video!!