Building a Personal Assistant using Agents (no-code) - Flowise Tutorial #8
2024 ж. 14 Мам.
11 200 Рет қаралды
#flowiseai #flowise #openai #langchain
Self-reasoning agents are one of the most important features of Flowise (and Langchain).
These agents use tools to achieve their goals and can be used in some impressive use-cases.
🙏 Support My Channel:
Buy me a coffee ☕ : www.buymeacoffee.com/leonvanzyl
📑 Useful Links:
LangSmith: www.langchain.com/langsmith
💬 Chat with Like-Minded Individuals on Discord:
/ discord
🧠 I can build your chatbots for you!
www.cognaitiv.ai
🕒 TIMESTAMPS:
00:00 - Intro
00:23 - Chains VS Agents
01:34 - Create chatflow
04:09 - Calculator Tool
05:21 - Langsmith
05:52 - SerpAPI Tool
07:44 - Chain Tool
11:12 - Retrieval Tool
17:45 - Outro
Note that the Conversational Retrieval Agent covered at the end of this video is being replaced with the OpenAI Tools Agent. Simply use that node instead of the CRA and everything should work the same. github.com/FlowiseAI/Flowise/pull/1991
Hello Leon. First, many thanks for your great work! As OpenAI Tools is currenlty marked DEPRECATING, I tried the NEW Tool Agent with success.
I built a basic personal assistant using AutoGPT, Chain Tool, and added an API Loader with OpenAI Embeddings etc. The information I got was a bit limited because of the Chain Tool. So a tool that improves upon the Chain Tool would be great. Btw, your tutorials are awesome, thank you.
As always, Thank you Leon.
Thank you ❤️
I would love to see future videos cover off Workflow Automation Agents, Data Analysis Agents, and integraion with other tools such as Python scripts that can scrape sites such as LinkedIn
Noted, thank you! I'm guessing you'd like to scrape LinkedIn to gather leads?
@@leonvanzyl Anything that can enrich a CRM wudl be great. Basically I would love to see Flowise (or any other opensource AI) used in more business contexts. Many AI's can do cool thingns for thin individual, but I would loev to see it employed at a departmental or organisatinal level.
Thank you Leon again... very informative and to the point! I just like your style 🙌
Dankie Manie
This tutorial on creating an AI assistant without coding is amazing! The explanation of chains vs. agents is very clear, and adding tools like a calculator or Google search makes it so practical. Excited to see more examples like the recipe generator! Great job!
Thank you very much!
Awesome tutorials, thanks Leon! I'd love to see a tutorial on how to use code interpreter / data analysis function use case with Flowise!
Thanks Leon for all the great videos in this flowise-playlist. You really helped me a lot understanding how this tool work and keep me motivated of try out things on my own. Really looking forward to watch the next videos.
Thank you for the awesome comment! You're welcome 🤗
Amazing Video! Thank you Leon.
Thank you!
Great video as always. Thanks.
You're welcome 🤗
great videos man! keep up the good work!
Thanks Rishab 👍
Great video @leon! 1. I´d like to see some "best practices" video when working with Flowise. 1. Also would like to see a video focused only on a dozen of useful tools, Not only existing tools, but tools that are useful and not already integrated in Flowise. Perhaps tools that extend functionality and integrates with other services out there, building your own tools guides etc.
i love how AI revived the thrill to build interesting solutions ❤ Thanks for the awesome product
I agree with you.
I would be definetely interested in a tuturial how to connect company data like e.g. email from outlook to automatically create answers to customers or distribute emails like to which department the email should be assigned. (Typically German long sentence construction :-) )
Thank you for this tutorial Leon. One topic that I would love to see in the future is ..how to make a simple API call ..and use the Response as a context for the LLM in answering questions. I can't seem to figure this out.. it should be straight forward.. but I'm having all sorts of issues with it.
Great idea. Will create a video on the API tools.
Another great tutorial. Thank you. This might be more trouble than it is worth, but I would love to be able to watch your typical video and then watch an extended version where you do more explaining about these various choices. Like the in-memory vector store vs Pinecone. This same video but when you switched to Pinecone you could talk about the differences beyond just using a pre-existing DB.
Dude, I would LOVE to do that. I'm just not sure about the demand. Maybe I should offer a paid course?
I don't know why you don't, people have had enough of you to know your knowledge is 100% good and professional. or at least create a members page where there are paid videos and you get renumerated for your work. I would pay.@@leonvanzyl
Suggestion: create a tutorial with 3 different agents acting TOGETHER. That's the challenge. How do I work in sequence with 3 agents or more?
Thanks man. Trying to figure out this KZhead thing. Appreciate your constructive feedback
interesting! sadly missing the source in the last output … but next step would be awesome to see: multi agent setup with hierarchy like in crew ai or langgraph :)
I'm not sure that collaborative agents in Flowise is possible (yet). Would be awesome though.
This please! If not in flowise, code it :)
@@leonvanzyl pretty sure, multi agent setup isn’t there yet. looking forward to it coming soon ;)
Great Video ! :)
Thank you
Would love to see a video about the different vectorstore options and especially which vectorstore to use if everything should run on the same server. so chroma or faiss, and regarding the on memory vectorstore when does it reset?
I must say Leon, the last two videos have been absolutely awesome. One error (which took way too long to find 😁, elementary error) was that I had a space in the name of my chef in the chain tool.
Thank you
Another excellent video! Congratulations! I would like to see some integration between Flowise and n8n if possible. Thanks.
Great idea!
Thank you for the professional and informative video. Pay attention that the Conversational Retrieval Agent is deprecated in the last version of Flowise
Yeah, they're replacing it with the OpenAI Tools Agent soon. I'll update the pinnned comment to reflect this. Thank you.
@@leonvanzyl Yes, I think it may even be the new Tools agent now as the OpenAI Tools Agent was saying depreciated. Thanks again for the great content
Great video as always. Could you make an Image to Text to Speech video with flowise?
"Leon, I really appreciate the high-quality content you are sharing here. We would love if you could create a detailed video explaining all the nodes, what their functions are, and how we can utilize them. The documentation on the website available is a little bit poor, so I think a video would be really helpful. Thank you!"
Thank you!
Leon, I have to say you're one of the best if not the best explaining stuff and showing us real world stuff. I simply love your content!! Could I ask you something? I you needed to create an agent with a lot of knowledge (around 600 pages), would that be possible in flowise? Or maybe separte the knowledge base in separate embeddings? Keep up the good work. Looking forward to seeing the next video!!
Thank you for the kind words. You can definitely that content into a vector store. I would recommend using a text splitter to chunk it though.
@@leonvanzyl Thank you Leon. What flow would you use?
Thank you sir for a great tutorial! I have one request. In Flowise, how do I make Llama3 running on Ollama, into a Tool calling chat model? Can I make it return structured output?
Really great and easy to follow video, again. Thank you! I haven't watched all of them yet, but one scenario that I don't know if you've already covered is using Flowise and RAG where the agent decides to use web search if not confident enough about the answer returned from the PDF. So for example like in this video if you would ask it about the newest released Tesla model, could you make it so that it figures out that the PDF didn't return a good enough answer and then still do the web search? (I know, best way is to build the agent in my own Flowise instance and try to figure it out. 🙂)
You could change the prompt template. Tell it to first look for the answer from the knowledge base before trying the other tools 👍
Bro, thank you very much, i paid for courses that had half of the quality of your free videos. If my startup became a sucess, I'll come back here and give you some prize haha
Haha, that is awesome! Just glad I could help.
Thank you! Impressive. Now only the real next step agent (kind of IFTTT or Zapier or own agents) is missing.
Oh wow, thank you VERY MUCH for the support! 🙏
Awesome video Leon (Loved the langsmith video too, awesome tool), not quite finished. Followed your video to the letter (up to where I am now14:48 minutes), using in memory vector store. Using my CV as an example. In the retriever tool my description is "Use this tool when asked questions about Wayne Bruton or Wayne." but I keep getting the following response: "Received tool input did not match expected schema", Really not sure what I am doing wrong? Any ideas?
Clearly you didn't watch the entire video 😁. I do discuss that error and solution 👍. You're using the wrong node.
Do you have any video to have a support agent based on business data using buildship?
Thank you very much, Leon, for your videos which greatly help me find solutions for my clients. They are always very clear and relevant. I would like to know if you have an idea that would allow working on lead generation with Flowise. Maybe a chatbot that could query Google or LinkedIn and would be able to create a list of prospects with their phone number and email, sent to a Google Sheet according to defined criteria ?
Thank you VERY MUCH for the support. It helps a lot. I'll try to create a tutorial on this topic 👍. I would combine tools like Make.com and OpenAI for lead generation, but I'm sure it's possible in Flowise as well.
Great explanation! Thank you! One question ... would it be possible to reuse the already existing recipe flow? By that I mean call it from this flow instead of having to recreate it block by block. Similar to what you would do when you create a library of functions in any programming language.
Absolutely! You could call the API of the Recipe Flow using an API Tool or Custom Tool.
@@leonvanzyl Ahh, you mean for example the way you would call the Flow from an application. I haven't explored APIs and Custom Tools yet, but I've seen that you have several videos on these topics. I'll watch them and see if I can put them in practice for this use case. Thank you very much!!
Great content as always! How can we customize the default chat window message "Hi there! How can I help?"
There doesn't seem to be a way to change it within Flowise itself, although I wouldn't be surprised if that changes soon. You can change it when embedding it in websites though. I'll release a video soon on website embedding that will show you how.
Is it possible to have more than 1 agent in the same flow? I would love to have different work assigned to specific agents. Lets say, one is good at researching and another one is good at creating a summary of all the research done. I know i can do that with chains but cant the same be done with agents? Or am I just not understanding the chains/agents correctly? 😞
Hi Leon, great video!! How much was the cost in OpenAi for these few questions?
OpenAI is cheaper than you think. Hard to say as these questions would have cost a fraction of a cent.
I am trying to build an agent that compares a JSON with user data from a database to a given final JSON format. It should first identify what values of the final JSON are missing and then needs to ask the user for the missing user data one by one (e.g age, name, email, adress etc.). Once the use has given all the answers the agent should store the updated JSON with the given answers in a database. Do you have an idea on how to start that challenge using Flowise? Thank you!!
This series is amazing, i would love to know how can i create a bot that makes appointments
Check out this video. kzhead.info/sun/ppimfa1-sGiPg58/bejne.html
What happens when we upsert another document while connected to pinecone free version. That is only using one pinecone index. Does the new document upsert overwrite the old document upsert? Does it append or add it on?
What if you had let's say a database of 40-50 documents, each with a description and a purpose, and based on the question or some other context from the chain, you wanted it to use THAT document. For example, I have a property management chatbot where I have google docs that have context info for each property. I am calling the langchain from another application, would i pass the URL of the google doc to a scraper tool somehow?
These tutorials are awesome. Thank you very much. I'm just having a little bit of problem making long PDFs work with Vector Store databases. When different parts of the PDF, parts witch are located far appart in the document, are related and share context, the agent is not able to pull both parts and gives incomplete answers or even wrong answers cause it doesn't consider all of the important parts of the document. How could we fix this?
Thank you! Increase the chunk size until the results improve. Also increase the k value in the retriever. Hope these help.
you did exceptional great Leon. please my question is how do i integrate my chatbot agent into an app like whatapp?
Thank you! WhatsApp is an extremely complicated process. Best to use a service like Twillio with Flowise. I might create a tutorial on this, but I haven't really found a cost effective and simple process yet.
Hi Leon, Could you please cover usage of Agent with API Chain Tools for making (GET and POST) API calls? Thanks
Will do 👍
Awesome! Flowise runs on top of LangChain js?
Correct. It uses the JS version of Langchain.
Thanks for another great tutorial, Leon! I'm pretty sure I'm following all the steps (and corresponding blocks), but my Retriever tool ain't working... I keep getting "TypeError: Cannot read properties of undefined (reading '_def')" Any chance you can help me, Leon?
I built again from scratch and it worked... I have no idea what caused the problem...
Glad you figured it out 😁.
@@leonvanzyl there must be some bug on the Flowise side, though. I've duplicated a working chatflow and got the same error again. I've deleted all the nodes and connected the blocks again. The error disappeared. I really don't know what causes this error.
Can this agent use 2 tools together/ concurrently
Hey Any help would be really appreciated I want to demonstrate the following Performance Metrics of a chatbot (not llm) Please guide how this can be done(any tools etc?) 1. Goal Completion rate: No of times goal was completed 2. Dissatisfaction Rate: No of times goal was not achieved 3. Fallback Rate: How many times the chatbot said it did dont undertand the query or did not give any response 4. Conversation duration: How long a conversation was in terms of time 5. Interaction rate: No of messages exchanged per conversation 6. Number of sessions: No of times users initiated conversation in a specific time period Please suggest any other metrics also that might be easier to implement than those i have mentioned
Hi Leon, Quick question. I notice, playing with openai assistants, I can go into the assistant(in openai) and change files as well as the prompt and it automatically updates my flowise. Which is pretty cool. Problem is openai assistants does not (yet) support streaming so has the appearance of being slow. Using your demo as per the video above, would it be possible to get a new document, embed it using openai embeddings and push to an existing index (which I am using in my flowise app). Doing all this external from flowise, would it work in flowise similarly to how it does with openai asistants?
Absolutely! You can update the index anyway you want. Flowise simply fetches documents from the index.
Hi, regarding the RAG, is it possible to set up rules so certain information can be accessed only to specific companies?
Yes, check out the API video where I show how you would be able to store and retrieve the data is seperate namespaces. For instance, you could have seperate namespaces for each use / client / company.
@@leonvanzyl Thanks a lot for you reply :)
I was wondering why you were using a conversational agent with a retrieval tool i.s.o using a conversational retrieval agent node in the first place? Also, could you shed some light on the difference between using these agents with ChatOpenAI as the model, and the OpenAI Assistant? BTW, around 11:30 you have a sentence that repeats twice ("answer questions based on you own data")
I like to build up the complexity when explaining things. Jumping straight into the retrieval node would then cause a bunch of people to ask me why I didn't just use the conversation node. I just can't win 😅. The OpenAI Assistant is actually not a model, but an entire framework which includes long term memory and RAG built in. Although Easy to use, it means vendor lock in. Some people prefer to build their own agents and select their own vendors.
A quick question, Leon... If I want my knowledge base to be multiple files... When does it make sense to split it to multiple Retrieval tools? Let's say I'm building a chatbot for a hotel. And the knowledge files contain: 1. Hotel FAQ. 2. Menu for the in-house restaurant. 3. A comprehensive PDF with local attractions (for planning local trips for guests). Would you create 3 separate tools (with 3 Pinecone blocks) or rather try to combine all the documents into a single Retrieval tool? PS. I know you're just a single person, so don't feel forced to answer. But I'll appreciate if you find time :)
This is a fantastic use case! I would use 3 seperate retrieval tools so that each tool contains a description related to the data being fetched like "use this tool when asked about the menu and restaurant". I know this might seem silly since the data are all stored in the same Vector database, but it does give you some flexibility. We haven't really convered this topic, but you can set a namespace when upserting docs to Pinecone. So in the retriever you can then only filter on the documents related to the menu as an example.
@@leonvanzyl I'm building this chatbot right now. Trying to leverage namespace and metadata in Pinecone as well. I could happily show you the results I'm getting (so that you have a project for another Flowise video). Let me know if you're interested.
Hi Leon .Great video. i followed the instructions for creating the LLM chain for unique recipe .See the error message i got from the input question what is for dinner ? iam in the mood for chicken {400 Invalid 'functions[1].name': string does not match pattern. Expected a string that matches the pattern '^[a-zA-Z0-9_-]+$'.}
I'm not familiar with that error. Try using the tools agent instead of the retrieval agent. Seems that the way Flowise is moving.
Hallo Leon, Dit is baie teid dat ek nie in Afrikaans praat en skryf nie. Ek was in Randfontein gebore, en dan in Krugersodrp geblei. Ek sal jy net frag, is dit moontlik om Langchain & Flowise op 'n bediener te installeer en meer as een persoon dit te laat gebruik. Hemel dit is nie maklik in Afrikaans om te skryf nie. Beter skakel oor na Engels. Great tutorial. great work keep it up. Just one question. Is it possible to set up a serve, with langchain and flowise on it. Then hook up to your own serve over the web or locally and have 2-3 people working on it in small office. I suppose it is possible, but suppose is aòways the srat of a F-up. So just asking. Thanks for your respone. Ciao from Italy.
Awesome comment 😁❤️. In the next video we'll have a look at deploying Flowise to the cloud so that your team can work on flows together 👍
hi nice guide , how i use llm studio for example if i wanna use. open source models and i dont wanna run via ollama
These agents require OpenAI as they use function calling.
@@leonvanzyl can it be done with llm studio when running open source llm like. llama2-38b/70b locally instraction model or minstral etc ?
Can node-red flows be used with flowise? Maybe in the capacity of a tool that could be used for specific things that node-red is better at than a LLM?
Hi, not familiar with node-red. Will look into it. Maybe someone else can assist 👍
When I saw flowise it reminded by of it. I hoped the projects might be somehow connectable. @@leonvanzyl
Whenever a new chat is made or tool is called, does the embedding process repeat again?
Embedding is only done once, when you upsert the docs.
Hi King Leon I have a question. It works with Google gemini?
"king lion" 🤣 Honestly, I haven't had much success when creating agents with anything besides OpenAI. You could try 😁
When I create a chatbot and implement it on my website using JS I cant increase the width of the chat window. Is there are hack/trick to do it?
This is not an embedding video 😁. I'll release a video on that soon, but there is no way to increase the width unfortunately.
Is there a tool within Flowise that can parse an XML feed?
Interesting question. Do you mean that the model should generate a result, and the result should be converted to XML? Output parsers are used to format the OUTPUT of the model. If you're referring to the opposite where the model received an XML structure as input, then no parsing is needed. These models are really good at understanding the contents of an XML structure.
@@leonvanzyl I was talking about the models receiving data from an online XML source as input. I did not see any tools to directly plug the XML URL as the source. I hope this makes sense. Thank you for your reply and for your channel!
The conversation retrieval agent does not support chat models with image upload - this is a real issue, how can I solve this?
Thanks for letting me know. I was just about to release a video on the image upload feature. I'll ask the Flowise team if this is a bug 👍
📝 Summary of Key Points: 📌 The video introduces the concept of creating a powerful AI assistant without coding, focusing on the difference between chains and agents. Agents can use reasoning to complete tasks by utilizing a list of available tools, offering more complex functionalities compared to chains. 🧐 The tutorial demonstrates how to build an agent that can answer questions, perform calculations using tools like a calculator, search the internet using tools like Ser API, generate unique recipes, and retrieve information from knowledge bases like PDF documents or existing databases. 🚀 By integrating retrieval tools, the agent can access and provide answers from specific knowledge bases, enhancing its capabilities for tasks like customer support or data retrieval. 💡 Additional Insights and Observations: 💬 Quotable Moments: Agents can leverage a variety of tools to enhance their functionality, from simple calculations to complex information retrieval tasks. 📊 Data and Statistics: The video showcases the practical application of AI assistants in various business scenarios, highlighting the versatility and efficiency they can bring to different tasks. 🌐 References and Sources: The tutorial emphasizes the importance of providing detailed descriptions for tools within the agent to optimize their usage based on specific tasks. 📣 Concluding Remarks: The video offers a comprehensive guide on creating AI assistants with advanced capabilities, showcasing how agents can be empowered to handle diverse tasks efficiently. By leveraging tools, knowledge bases, and retrieval mechanisms, businesses can enhance customer interactions, streamline processes, and improve overall productivity. Generated using TalkBud
Oh wow, this is cool!
How much value in a free video 🙏🙏🙏
Thank you!
Flote, your hearts always make my day! Thank you!
@@leonvanzylyour work helps me so much. Thank you ! From France ;)
Can we talk to MySQL?
Absolutely.
Why are my comments being removed?
I never delete comments. Perhaps they violate YT guidelines, who knows 🤣
@@leonvanzyl The first time it happened to me, I wrote 2 comments and they disappeared. And I don't know why? If YT deletes they should write why. No matter. I asked about the tutorial about LlamaIndex and what do you think about Bubble for website integration?
Hehehe, perhaps.@@leonvanzyl
@leonvanzyl I have a issue its giving me a error 400 invalid model ID
Which chat model are you using?
@@leonvanzyl hello sir thank you for reaching out I’m using GPT 4 turbo 1106 preview
@@leonvanzyl also sir I saw that the bubble on the chatbot is with another color you have a tutorial on how I can modify the UI of the chatbot change the background color and bubble color?
@leonvanzyl I am having a issue with the yarn build it says that there’s a error