KZhead
Ғылым және технология

Fine-tuning a CRAZY Local Mistral 7B Model - Step by Step - together.ai

All About AI
2024 ж. 16 Мам.
29 983 Рет қаралды
Жүктеу

Fine-tuning a CRAZY Local Mistral 7B Model - Step by Step - together.ai
👊 Become a member and get access to GitHub:
/ allaboutai
📧 Join the newsletter:
www.allabtai.com/newsletter/
🌐 My website:
www.allabtai.com
In this video i fine tune a Mistral 7B model with a self generated dataset. We will go step-by-step over on how to create the dataset, convert to JSONL, upload it, start the fine-tuning, download the model, convert the model to .gguf and test it.
GH llama.cpp:
github.com/ggerganov/llama.cpp
Reddit:
www.reddit.com/r/LocalLLaMA/c...
00:00 Local Fine Tune Intro
00:18 Flowchart
01:14 Create Mistral 7B Dataset
05:59 Check Dataset
06:56 Upload Dataset
07:26 Start fine-tune job
10:58 Convert model to gguf
12:39 Testing our fine tuned model
16:09 Conclusion

Пікірлер

  • You explained in one video what several courses struggle to explain in dozens... Take a lot!

    @luisguimaraes7348@luisguimaraes73483 ай бұрын

  • I’ve learned more from your channel than any other on YT. Thanks, Kris!

    @GeorgeVuckovich3@GeorgeVuckovich33 ай бұрын

  • 2:32 I remember when 25 examples was more than enough to get started with fine tuning. And now less than a year later, 300+ examples is the minimum. It’s amazing how much I’ve learned from this community and your videos. Thank you 🙏

    @OutstandingLife@OutstandingLife3 ай бұрын

    • 25 examples qualifies and few shot prompting, not finetuning.

      @dmy_tro@dmy_tro3 ай бұрын

  • Thanks, Kris, for teaching and keeping us up-to-date in this crazy AI wave! Practical code tutorials are always helpful and appreciated.

    @seankernitsman6055@seankernitsman60553 ай бұрын

  • Absolutely loving the content on this channel! Your insights on using AI for fun projects are both informative and inspiring. Each video feels like a new adventure into the potential of AI, and your clear, engaging explanations make even complex topics easy to understand. Can’t wait to see what you’ll explore next!

    @darrylrogue7729@darrylrogue77293 ай бұрын

  • Thank you! I'm definitely going to try this. It's the first time I see the process somewhat clear!

    @STRIKE01@STRIKE013 ай бұрын

  • Love your work man, learnt so much being a member!

    @silentphil77@silentphil773 ай бұрын

  • It's amazing at the way you explain the concepts in a super simple manner. Kudos to you. Keep the banging content coming our way :)

    @saahilsachdev@saahilsachdev3 ай бұрын

  • Doing amazing work, Kris!

    @AzureRaven6969@AzureRaven69693 ай бұрын

  • I really new to this but maaaaan I enjoy the hell out of these videos. I look forward to making my own as soon as I can

    @Co0k1em0n5t3r@Co0k1em0n5t3r3 ай бұрын

  • your videos never disappoint! Keep it up with AMAZING content.

    @ghosting-cm7zf@ghosting-cm7zf3 ай бұрын

  • Talk about an AI with attitude. Thanks for the walk through. I haven't used fine tuning and could use the additional information.

    @watcher1326@watcher13263 ай бұрын

  • Always informative and enthusiastic!

    @openminddream@openminddream3 ай бұрын

  • Amazing. Will try it out today for finetuning

    @pushpaaji@pushpaaji3 ай бұрын

  • Love being a member, Kris! Always overdelivering value. This video was a great overview of fine tuning your own model. Simple enough for me to understand. 😂

    @JasonBevel@JasonBevel3 ай бұрын

  • Thanks for the detailed video.. fine tuning seems worthwhile!

    @nexuslux@nexuslux3 ай бұрын

  • Thanks for the great content and repos. Super helpful content.

    @djjeffgold@djjeffgold3 ай бұрын

  • This is awesome! Great job!!

    @PhunkNPharaoh@PhunkNPharaoh3 ай бұрын

  • Another great video, thanks Kris!

    @carstenli@carstenli3 ай бұрын

  • You are doing really great and we are learning a lot by you

    @vaibhavmishra1100@vaibhavmishra11003 ай бұрын

  • Thanks for everything ❤.

    3 ай бұрын

  • You know from the value that you produce in your channel. I might just purchase whatever paid content you have. Truly amazing, think about starting a Skool community man. You're great.

    @harristengku7153@harristengku71533 ай бұрын

  • Thanks! It was very insightful.

    @Kasyx@Kasyx2 ай бұрын

  • This is the most straight forward fine tuning tutorial on youtube ever!

    @ilhamNurFachri@ilhamNurFachriАй бұрын

  • Thanks for sharing Kris :)

    @AG-yy3fq@AG-yy3fq3 ай бұрын

  • Amazing video, Kris!!

    @ygoralves4664@ygoralves46643 ай бұрын

    • Hey there! I am thinking of signing up and was wondering, how's the membership going for you? Have you had a chance to implement his code? If so, is it working as expected for you?

      @ForTheEraOfLove@ForTheEraOfLove3 ай бұрын

  • Awesome Video, thank you for you content.

    @josefsteiner8616@josefsteiner86162 ай бұрын

  • Interesting, but the time it took to train on that small dataset is.. a lot. As always, thanks for presenting new stuff :).

    @blacktyg3r@blacktyg3r3 ай бұрын

  • Woot Woot!! Thanks for the fantastic Video! Oh, and for the pick 5 😉

    @ejh237@ejh2373 ай бұрын

  • Value for the money. Thanks Kris :)

    @avgplayer@avgplayer3 ай бұрын

  • Things are starting to get more and more local baby! 😁

    @eatingalienbrains@eatingalienbrains3 ай бұрын

    • locals forever! :D

      @FallenChapter@FallenChapter3 ай бұрын

  • Great Info!

    @weekendmakeit7760@weekendmakeit77603 ай бұрын

  • This looks awesome actually

    @ShikariNagasa@ShikariNagasa2 ай бұрын

  • Looks good!

    @Ryan.Youtube@Ryan.Youtube3 ай бұрын

  • Thanks great job :)

    @Daweiro@Daweiro3 ай бұрын

  • Leaving a comment for the youtube algorithm. Thanks for showing an A to Z with this!

    @johnsummers101@johnsummers101Ай бұрын

  • This is so cool! I'm gonna make one of these for each of my AI Vtubers. And together is such a great platform, especially with the free $25 in credits!

    @digitalasylum369@digitalasylum3693 ай бұрын

  • OMG, I was just surprised multidimensionally XD I mean everything is great man. content is beyond rich and email part and answer was the most hellarios moment of my past 2 weeks.just tnx XDDD

    @saeedasgaritehrani@saeedasgaritehrani2 ай бұрын

  • I would love to see you explore front end options or adapting modalities with local models.

    @joepearson7928@joepearson79283 ай бұрын

  • Thanks!

    @DavidChristal@DavidChristal3 ай бұрын

  • Nice technical video! Can you dive into the topic of LLM evaluation? For example we fine-tuned model using approach like you showed. How to check the quality of results?

    @trashchenkov@trashchenkov3 ай бұрын

  • super cool

    @luisguimaraes7348@luisguimaraes73483 ай бұрын

  • Great info

    @abs9994@abs99943 ай бұрын

  • Nice, I’m going to do this tomorrow…

    @Canna_Science_and_Technology@Canna_Science_and_Technology3 ай бұрын

  • Most intersting Channel and my english is Bad but hé make things look very soft thanks alot

    @carthagely122@carthagely1223 ай бұрын

  • Hello Sir! You say at 12:40 that there is a video showcasing how to set things up. I failed to find it, can you provide a link please?

    @jonatan01i@jonatan01i2 ай бұрын

  • very interesting work! thanks for the lessons. I wonder how much of this programming, if not all of it, can be done autonomously. Probubly an expensive agent task, but curious to push the envolope of AI's capabilities. Fully autonomous assistant populations who live in the internet as we live in societies. Fascinating and inevitable.

    @elidelia2653@elidelia26532 ай бұрын

  • That's so hilarious, please share this model with us, I would like to make some tests

    @MugenMangekiou@MugenMangekiou3 ай бұрын

  • can you make a video about sentiment? I have a dataset and i want a model to give a score between 0 to 100 (bad to good) about each statement.. I have no idea where to start.. thank you.

    @hqcart1@hqcart13 ай бұрын

  • How can I fine tune model on own codebase? Is it possible to make dataset from a code?

    @DemiGoodUA@DemiGoodUA2 ай бұрын

  • Anybody else getting 401 errors when uploading the jsonl files? Tried it in powershell, CLI, python, know my API key is set correctly, inference models work perfect.

    @nickcochrane6568@nickcochrane6568Ай бұрын

  • what are the specs of your pc?

    @junmagic8847@junmagic88473 ай бұрын

  • Nice video :)

    @nenesh66@nenesh663 ай бұрын

  • Hmmm, so you call a model "local" while you already shared your info in that .jasonl with the AI Platform using API Key!?!? Am I missing something here!

    @positivevibe142@positivevibe1423 ай бұрын

    • @@VioFax Right. It's just kind of negates the the notion of 'locality' and the reason why people/users go with offline/local venues.

      @positivevibe142@positivevibe1423 ай бұрын

    • @@positivevibe142yeah I want local so that when the grid goes down and takes the internet with it, I can still learn to build and repair things and grow seeds and hunt food. With the knowledge from local LLMs. Not sure what you guys are doing that you have to hide it so much? Your AI gf get a little frisky? 😂

      @p1nkfreud@p1nkfreud2 ай бұрын

    • @@p1nkfreud Funny ... lol ... but really this is a very crucial thing! Like for a small business, scientific researcher, medical facility, financial agency, legal firm, laboratory facility, ....etc ..... believe or not the list longer than you can imagine ..... Secret Boyfriends lab research 😁

      @positivevibe142@positivevibe1422 ай бұрын

    • ​@@p1nkfreudset aside about $10K if you want to train locally

      @Joooooooooooosh@Joooooooooooosh2 ай бұрын

    • How else would you fine-tune the model? Some of us don’t have massive compute power that requires a substation to power it. 😅

      @len5499@len54992 ай бұрын

  • I subscribed to the channel but i don't get why it's not letting me in in the github pls someone explain) thx I also tried signing up on the website but i go the 40 promps and tha't it

    @greatsarmad5147@greatsarmad51472 ай бұрын

  • Cannot find code for the data set maker. I wish u would have gone over how to build that part a bit more cause I can’t find anything about using lm studio to make datasets

    @spencerfunk6697@spencerfunk6697Ай бұрын

  • The future is local!

    @cognivorous1681@cognivorous16812 ай бұрын

  • Hy bro where I started in Ai

    @subhansajid1435@subhansajid14353 ай бұрын

  • I mean the speech was actually pretty cool I wonder how you were able to do it

    @vivekkarumudi@vivekkarumudi3 ай бұрын

  • How much it costed approx. 9h of training in this example? Just for a reference please.

    @TrevorDBEYDAG@TrevorDBEYDAG2 ай бұрын

  • Can you please link the python script to convert dataset?

    @tr1pod623@tr1pod6233 ай бұрын

  • Sometimes talk to yourself - grandmother wisdom

    @andreasrosenfelder9094@andreasrosenfelder90943 ай бұрын

  • I can't wait until everything is completely local. They can't legislate what is on your local hard drive! Hope I'm one of the lucky winners! :D

    @CypherDND@CypherDND3 ай бұрын

  • I followed this tutorial but LM Studio spits out all the fine tuned data even when I just type something simple like "hello".

    @webnomad1453@webnomad1453Ай бұрын

  • Similar to your text datasets. I had luck training using axolotl, and it took around 1 hour using two A40 instance with around 40gb vram. On vast. For around 1 usd per hour. I have not yet tried Together, but Together seem simpler to configure than axolotl.

    @simonstrandgaard5503@simonstrandgaard55033 ай бұрын

  • hilarious😂

    @DannyGerst@DannyGerst3 ай бұрын

  • Which u luck for what u doing 👍👋

    @jojojkk1335@jojojkk13353 ай бұрын

  • First you said you will use the reddit as dataset but later you created the dataset using the Mistral API itself. Don't know what I am missing here

    @yashinshekh802@yashinshekh80219 күн бұрын

  • You can build a website for people to hire for data training, where you take a portion of the money. You assist them in training and return trained files in quantized format. We have a high demand for personal assistant training, especially for private document sets for individual purposes. Therefore, I think you should consider this. We're not skilled in coding, don't want to complicate each step, but we have large PDF, DOCX documents that need training, and we need the output to be quantized to the lowest "Q2" level to run on popular devices like iPhone or Android, not large-scale quantum versions. We focus on the popularity of handheld devices, as not everyone owns a server or wants to pay a large amount for data running through APIs. I can offer prices from $20 to $50 for a 100MB document set. I hope you'll consider offering a rental training model soon.😊

    @khoahocgia7671@khoahocgia76713 ай бұрын

    • This is a great idea - creating this soon. I'll keep you updated :)

      @duckyblender@duckyblender2 ай бұрын

  • hi , whats the difference between lms studio and oogabooga???

    @laif9857@laif98573 ай бұрын

    • Nothing lmstudio is just easier to use

      @DoppsPkin@DoppsPkin3 ай бұрын

  • No way that I will upload online to train my data . How can I train locally ?

    @gerykis@gerykis3 ай бұрын

  • Hope I win the credits😅

    @maheshBasavaraju@maheshBasavaraju3 ай бұрын

  • To generate a passive income

    @subhansajid1435@subhansajid14353 ай бұрын

  • You just ruined a good video by mentioning a Nigerian prince.. like can’t a video be just simple and clear without mentioning a country..

    @danielejiofor3493@danielejiofor349317 күн бұрын

  • The best Channel for AI for me. On Point, not too much and every shinything and i learn better english... Thnaks for all @AllAboutAI 🙏

    @David_Mederake@David_Mederake3 ай бұрын
KZhead