Llama 3 8B: BIG Step for Local AI Agents! - Full Tutorial (Build Your Own Tools)
Llama 3 8B: BIG Step for Local AI Agents! - Full Tutorial (Build Your Own Tools)
👊 Become a member and get access to GitHub and Code:
/ allaboutai
🤖 Great AI Engineer Course:
scrimba.com/learn/aiengineer?...
🦙 Easy Local RAG Ollama:
github.com/AllAboutAI-YT/easy...
📧 Join the newsletter:
www.allabtai.com/newsletter/
🌐 My website:
www.allabtai.com
Links:
ollama.com/download
huggingface.co/sentence-trans...
In this video I create a AI Agent that can controll tools using APIs and internal functions. Leveraging local llms like llama 3 with function calling can be very powerful and highly customable. Full tutorial on how to create local llm tool calling.
00:00 Llama 3 Agent Intro
01:47 Llama 3 Function Call Code
04:52 How the function calls works
11:22 Llama 3 Agent Test 1
13:46 Creating a llama 3 tool / function
16:08 Llama 3 Agent Test 2
16:47 Conclusion
Playing with Llama 3 on Ollama and it is bonkers good
thank you for the break down it will help us to create our own specialized agents
thnx for tuning in =)
Impressive, very nice. And speed of this is insane.
thnx :) yeah its great, really happy for these 3 models from meta
Great video! Thank you very much for the explanation.
Thank you for sharing! Awesome! 🔝🔝🔝
Outstanding demonstration, thank you. I would like to make it possible to add new tools using a yaml or json configuration file so you don't need to hard-code anything. It would also be nice to have the answer returned immediately instead of having to prompt it to check the retrieved context. But all in all this one of the best and most practical open source AI demos I've seen.
Great video I am super excited about being able to use function calling with a smaller model.
yes me 2 =) and thnx for tuning in!
Yes me too, especially with a local one
Useful information. Thanks!👍👍👍
one of the best videos ive seen on this topic maybe the best! keep it up broski!
Thank you 🙏 great breakdown
thnx for tuning in mate =)
Amazing content. What of your videos can I watch prior to these steps to understand how to get Ollama?. new Argy sub.
Is the model fine tuned in anyway to use the function calling convention (tags + json), or is it solely based off the system prompt? If the later, I am mighty impressed for a 8B model.
Something that is not clear for me is why we should use the open ia API in the agent, was it not possible to only run the entire model locally ?
Everything is running locally. OpenAI we use because the OLLAMA API is compatible with OpenAI, so we point to our Ollama as the address
yeah like @growdelan9855 said =)
I have a same question!
Great video😮😮
That's amazing! What is the difference between using LangChain and not using it? Thanks!
I love the content. I have an idea for a video and this is something a lot of people could use. I’m terrible at taking meeting notes. A local model that could listen using the laptop mic and summarize in person meetings would be amazing.
Nice video!
Thanks for putting this together! What would you recommend as a good budget/starter setup for training Llama 3 8B? a RTX 3070 or better? a decent 8-core cpu like 13600kf ? 64gb ram i assume? I'm not in any sort of rush so I'd prefer not to rent the processing and purchase my own equipment over time.
hello, thnx for tuning in :) im no hardware expert, i run it great on a 4080 with 16GB vram. you can maybe ask in /r localllama
@@AllAboutAI thanks. yeah im in their discord right now browsing around.
How does it compare to the likes of Dolphin-Mistral which has the bigger context window/code training?
I cant host 70b locally or even 8b, i need to use an llama3 hosted api with agents that crawl many urls, can you suggest a setup that would work for that
I've just joined the community. I'd like to use LM Studio instead of ollama as ollama doesn't like windows. how can I adapt the local rag with LM studio, and I would like to create some agents. Thank you.
Where is the code for this Vid?
I would also love some code to base on
Same. This video being the reason I joined 😅
Awesome. Are you sharing the github repo anywhere?
I think you have to become a member of this channel, but IDK how that works. Maybe just subscribe.
@@wurstelei1356 I paid to become a member to get github access. Once you join, the github access is 404'd, and lots of other members are having the same issue and say there has been no response for many days - EDIT - All About AI got back to me after a day to ask for a github account name - so it worked out in the end
This is amazing, thanks for sharing.
what quant you using?
ollama.com/library/dolphin-llama3, Q4 i think this is
I would do these things too, but scraping is against googles term of use and almost every site has something against scraping. Could potentially be risky for you
When I use llama 3 8B on ollama or LM Studio, it is much dumber than on OpenRouter. Even after resetting all parameters to factory and loading the llama 3 preset. Even with the full non-quantized 8-bit version on LM studio.
Mr. Ai, please create prompt installation
Thanks for your video! Any background music is very annoying in tutorial videos.
Do you know hardware requirements for 70b model (locally)?
It depends on which quantization you use but it needs over 40GB. You can run it all in RAM or devote some of your GPU VRAM to it. I run it locally and devote 10GB of VRAM from my 12GB 4070Ti.
By the way I use LM Studio on both Windows and my Mac to run llama 3 70b.
Hi, I just joined, how can I get the git for this code?
How is Llama 3 compared with Mistral. Isn't this a risk to be dependent on Facebook one more time ?
its great, no risk because you can run the model offline if you want to
Have tested llama 3 70b Instruct with gpt-pilot. It does not work. The model is too stupid to work like GPT-4-TURBO-PREVIEW.
no i have not yey, thats strange tho
cant help thinking if you could do a video to run Microsoft UFO on local computer using an open-source model with vision? That will be quite promising and one step closer to AI PC
What’s Microsoft UFO? I looked online but I’m still not quite sure what it contributes to an AI assistant
Just did some more digging, it actually sounds really interesting and could have huge potential, plus it’s open source!
The word "so" was used 426 times in this video! 25 "so" per minute! So, its look like Chris very loves the word "so". Hahaha)))
Are you sure it wasn't sew or sow? Just thought I'd plant some seeds... and got myself in stitches (sorry)
Great video, how to download the code?
need to be a subscriber to get access to all code of the channel
@@MrMagicmars I have become a subscriber. How can I get the code? Thank you very much for your help.
I have it up on my community GitHub now for my members as a perk :)
@@AllAboutAI Thank you for your reply. How can I find the link? I don’t see the link on the homepage.
@@MrMagicmars I paid to become a member to get github access. Once you join, the github access is 404'd, and lots of other members are having the same issue and say there has been no response for many days - EDIT - All About AI got back to me after a day to ask for a github account name - so it worked out in the end
Ricky Bobby.
Use a hacker thief's AI that has no inventor as an "agent" slave to do your bidding... what could possibly go wrong.
This is what always gets me about function calling, unless you intend to make the endpoint available to the public, it would be far easier and faster to just expose the function call. So instead of asking the AI to search, you just call the search command directly. Using AI to figureout what function to call and the needed parameters, is like using a flame thrower to light a candle. It only makes sense if the user doesn't understand function calls.
🤖 (00:00) **Intro to Llama 3 8B and its capabilities** 🧠 - Demonstrates the agent's ability to search Google, extract information, and send emails. 📧 - Highlights the impressive instruction-following abilities of the model. 🛠 (02:00) **Code Deep Dive - Setting Up Functions** - Explains the functions used in the system, including search Google, send email, and check context. - Shows how to add custom functions, like write to notes. 📝 🧠 (04:21) **Understanding Function Calls** 🤔 - Breaks down the process of how the AI agent interprets user input and triggers function calls. - Explains the role of secret instruction notes and the parse function call function. 🕵♀ (07:18) **Surveillance System** 👀 - Discusses the surveillance part of the system that monitors the AI's output for function call tags. 💻 (11:21) **Showcasing the System** - Demonstrates the system in action by performing tasks such as searching Google for AMA models and extracting information from context. 📝 (13:33) **Adding a Custom Function (Write to Notes)** - Walks through the steps of creating and integrating a new function that writes content to a text file. ✅ **Benefits** ✅ Shows the potential of local AI agents with Llama 3 8B. ✅ Provides a detailed explanation of the code and logic behind the system. ✅ Offers a tutorial on how to set up and customize your own functions. ❌ **Cons** ❌ The video is quite long and technical. ❌ Assumes some prior knowledge of AI and coding concepts. ❌ The explanation can be a bit verbose at times. **Key Takeaways** 🔑 - Llama 3 8B is a powerful language model that can be used to build local AI agents. - The system uses a combination of functions and instructions to perform tasks. - Users can customize the system by adding their own functions. **Summary** The video provides a comprehensive tutorial on building a local AI agent using Llama 3 8B. It covers the code, logic, and functionality of the system, as well as how to add custom functions. While the video is technical and assumes some prior knowledge, it offers valuable insights into the potential of local AI agents. **Verdict: Like 👍** (Despite the cons, the video's educational value and insights make it worth a like.)
Any way to get the code in git?
I have it up on my community GitHub now for my members as a perk :)
@@AllAboutAI I paid to become a member to get github access. Once you join, the github access is 404'd, and lots of other members are having the same issue and say there has been no response for many days - EDIT - All About AI got back to me after a day to ask for a github account name - so it worked out in the end
Full tutorial but code is pay walled, reminds me of "open"ai
Now we know which tutorials to avoid! This one was mostly clickbait anyway.
anyone charging deserves blocking
It's the concepts and principles that are the most valuable -if you want free code you can get that wherever, doesnt mean it's going to be valuable. But if you really want his code then write the code to get it - rip the audio track and transcribe. Extract video frames and put through an OCR. Take the raw outputs from the audio and the raw ocr and get an AI to put it together. Or ... you learn the principles and you write your own and learn... or if you want copy pasta and cant wait... pay the guy for his hard work.... everyone has bills.
@@bennie_pie if I need to break down why this is a dumb and greedy move then you should probably be watching other videos
@@jeremycollins6682 To be fair you made the first daft move - a comment creates engagement which increases reach ... probably a few more views for the creator. It wasn't a reply aimed at you to be truthful. It would be silly of me to think that I could change your mind (nobody changes anyones mind arguing online, at least not easily) it was there for balance for the person who did want copy pasta but could be convinced that there is value in learning, paying a fair price for hard work. Not unreasonable - oh and my suggestions re ripping the code out the video... clearly impractical and ridiculous and would take longer than just learning it. No such thing as a free lunch. I'm not a member but he seems to be doing allright...so all good here, but feel free to reply further...
great video, but please remove annoying background music
They sell the idea that these models are extraordinary, with exceptional performance, capable of answering various questions, etc. Then you download the model and when you ask it to perform a ridiculous task, such as passing a set of a few sentences and asking it to return the one that most relates to what you said, then the model either hallucinates or gives a wrong answer. How can you trust something like this to perform everyday tasks? Or how can you give it freedom to execute actions on your operating system? Unfortunately, we are still dependent on large companies with models trained with billion-dollar investments.