Chatbots with RAG: LangChain Full Walkthrough
In this video, we work through building a chatbot using Retrieval Augmented Generation (RAG) from start to finish. We use OpenAI's gpt-3.5-turbo Large Language Model (LLM) as the "engine", we implement it with LangChain's ChatOpenAI class, use OpenAI's text-embedding-ada-002 for embedding, and the Pinecone vector database as our knowledge base.
📌 Code:
github.com/pinecone-io/exampl...
🌲 Subscribe for Latest Articles and Videos:
www.pinecone.io/newsletter-si...
👋🏼 AI Consulting:
aurelio.ai
👾 Discord:
/ discord
Twitter: / jamescalam
LinkedIn: / jamescalam
00:00 Chatbots with RAG
00:59 RAG Pipeline
02:35 Hallucinations in LLMs
04:08 LangChain ChatOpenAI Chatbot
09:11 Reducing LLM Hallucinations
13:37 Adding Context to Prompts
17:47 Building the Vector Database
25:14 Adding RAG to Chatbot
28:52 Testing the RAG Chatbot
32:56 Important Notes when using RAG
#artificialintelligence #nlp #ai #langchain #openai #vectordb
Excellent tutorial with clear comparison before and after RAG. The Python code walkthrough is useful.
very informative video, got understanding of RAG, and LangChain as well, successfully created my very first RAG based chatbot, Learned a lot 👍
Amazing James! Really Helpful Material
Nice glad you ran over this I’ve been stuck on part of my RAG but I’m sure your going to clear that up! 🎉thanks
hope it helped!
Great video! Congrats 👏 Excellent speaking capability, keep it up!
Thank you for your videos. They are all awsome. Could you please also talk about how can we extract more granual meta-data from our documents (like page number for example), and how we can force an agent to add the meta-data to the final answer it provides?
🎯 Key Takeaways for quick navigation: 00:00 🤖 This video demonstrates how to build a chatbot using retrieval augmented generation (RAG) with OpenAI's GPT-3.5 model to answer questions about recent events or internal documentation. 02:58 💡 Language models like GPT-3.5 can sometimes provide inaccurate or hallucinated information because they rely solely on their training data and don't have access to external knowledge. 08:16 📚 Chatbots can use external knowledge or "Source knowledge" to improve responses. This can be done by inserting context or documents into the prompt. 16:09 🔍 Retrieval Augmented Generation (RAG) combines parametric knowledge from the model's training with source knowledge from external documents, allowing the model to access and update information. 19:33 🧠 Pinecone is used as a vector database to store embeddings of external documents for RAG, allowing for efficient retrieval and updating of knowledge. 23:44 🚀 Embedding documents for retrieval in RAG is typically done in batches to avoid overloading the model and network with too many embeddings at once.Is there anything specific you would like to know more about or any questions you have regarding the key takeaways from the video transcript? 24:25 📊 Preparing external documents for retrieval with Pinecone should consider batch size limitations to avoid problems. 26:00 📄 Initializing a Vector database and embedding text chunks for retrieval with Pinecone's Lineage. 27:07 🤖 Using source knowledge from external documents to augment prompts and improve chatbot responses. 28:43 🧠 Augmenting queries with source knowledge for chatbot responses, demonstrating improved answer quality. 31:14 🛡️ Retrieval Augmented Generation (RAG) can significantly enhance the retrieval performance of chatbots by utilizing external knowledge. 32:23 ⏳ Implementing RAG with a simple approach but noting that it may not be suitable for all types of queries. 33:47 📈 Benefits of using RAG include improved retrieval performance, citation of information sources, and faster response times. 34:43 💰 Considerations when using RAG include token usage, cost, and potential performance degradation when feeding too much information into the chatbot. Made with HARPA AI
I love your statue in the background
Hey man, love ur vids. Great quality
thanks a lot buddy, keep on the hard work you help me so much!
Would really love to see you focus more on opensource
Nice video. Any tips for how to handle the "Hi, how are you?" case where you don't want it to do the retrieval part? I'm thinking there must be a way to limit retrieval results based on a similarity score threshold or something. Thanks!
Great video James!
How would you do rag and add coverstational history as well?
Hi, your video was great . I have a question. What is the advantages to use pinecone over Faiss to store your vectors?
Very well described!
Is there any technique that can be used to verify each sentence of the response with the given context? Except feeding it to the llm as that could have hallucinations and consumes openai tokens
I think the initial answers from the AI about string theory were correct. I'm not an expert either, but it seems like they were. Also string theory is largely present in media, popular culture... so it would make sense for the AI to be well informed about it :)
would this work with other LLMs from huggingface ? and other embeddings like Chroma ? if so , what LLM would you recommend ?
Great walkthrough as always. My auestion would be is it beter to use LLM chain like conversational chain or chatbot is better for RAG? also RAG Agents, althoug they run on huge amaount of tokens.
@jamesbriggs In production - do you tend not to use the Conversation Retrieval Chain? In the example you posted- how would you add memory into it?Great videos as always
Great lecture
excellent tutorial
Oh my! Tate is now a developer in the matrix.
the matrix got me in the end
@@jamesbriggs 🤣🤣 Nevertheless, Thanks for the efforts you put in to create awesome contents.
Hey, Wonderful Explanation, Could you please help me understand one thing, at 30:35, Jupyter Cell No - 36, The response from augmented prompt (about speciality about Llama 2) isn't been appended to the messages. How come in the next run when trying "Safety measures" prompt, model already has some information about the Llama 2 when we haven't actually given the model the chat history??
SystemMessage, AIMessage, would these work for LLAMA2 too? Since it's syntax requires [INST] and [/INST]
Thx for the video :) can you update your vector database by a few lines ( if you want to add data to your knowledge base) automatically by running a python script or something like that?
Is there a working QA model that knows the Langchain documentation? The official ones don't seem to work
¿Se puede usar para el rastreo recursivo como base de conocimiento de una dirección URL?
This is great. Are there any references for chunking the source docs and marking the chunks with metadata that you would recommend?
yeah for sure, check out this video kzhead.info/sun/mNWIltZseHmCnnA/bejne.html :)
@@jamesbriggs
A minute after being uploaded, I am watching
💪
Can you please make a video with an opensource free vector database? Like chromadb, for example. I love your videos ^^
Very helpful video. Is this doable with Llama-2 instead of using ChatOpenAI() function from OpenAI? Basically I am trying to do the same with Llama instead of using OpenAI's GPT model
ok generally a good talk.... Similar to the q&A with Context... We need to understand that the LLM < Longterm Base memory and the LocalDB is also An Augmentation to Updated Longterm Memory , We could say its a form of Short Term Memory as it is Updatable and Appendable ... the chat memory is the Working Memory (disposable unless stored into Short Term Memory DB) .... In a well designed model the short term memory can be used to FINE TUNE the LLM ie Updating the base knowledge in some form of interval to keep the base memory upto date, at this point new methodologies which use the " Question/Instruct - Context - Answer/Response) structure... the new base model can be redistributed ... for users ... Subsets of the short term DB can be then deleted and Specialised Domain knowledge HELD and NOT UPDATED Into the base Model (only desired fine tuning sets to perform specific instructs (such as diagnosis)...or ROLE PLAYING.... only a small subset needs to be fine tuned to allow for the new ability or skill.... where the actual knowledge can still be held locally in domain based knowledge vectorDB.... One thing people have seem to have forgotten about vector DB is that they are still databases hence can be structured ! and even relational ... hence converting existing databases to vector databases with structured based querys available for use by the search model (new instruct tuning required - for each query added)... ie the model could be used to search a specific patient history and use as context.... COnnection to the real world scenario of diagnosis already existing in the domain ... enabling for rich search and querying and diagnosis as well as possible discussive sessions with the "doctor or nurse" utilizing the system etc... SO the use case i described could occur... We are getting close to "Machine that have rationality as humans rationalize" (think as humans).... with function calling and embedded (function calling) we get to the "machine which act like humans" with a large action model... ie: functions have been prestored in the training dataset, with executing pathways for completions... for specific actions... ie calculate A+B it should have access to a calculate function ! to be executed if possible ...or returned as the response to calculate ... given a response the llm system its connected to can execute in pythonnet.... possibitys endless .. Tutorial ReView: Problems: today no actual code was displayed for the full models .... As well as each point you discuss should end with enforcement : IE an image of FULL STOP! .... hence not easy to follow your train of thought or end point ! this you need to improve as the content is fine but the presentation is not quite right! ...(its Full stops ! usually done with voice intonation and pauses bro! perhaps the editing has removed these important aspects.... as well as often discussing things which could have also been displayed on the screen to enforce the idea.... switching back to the original diagram being discussed! ) hence sometimes it seems though your randomly speaking and drifting off.... i would hate to have you as a teacher when you get older !!!! it would be unbearable ! (knowing you have the knowledge i need but cant get it from you!) ..... Tutorial Request: 1. How to run multiple models at the same time; Reason : when creating a multi agent system ... we have specific model which are specialists and other which are generalists.. ie the generalists can be prompted to act as a specialist but cannot outperform the specialist on its specialization... hence requiring multiple models ... 2. How to merge models : (LLAMA-Based GGUF) ? Some models have vastly improved and the keeping of multiple models often is not viable so merged models can provide a way to combine knowledge bases and skills. 3. How to finetune a model to have a larger input context ??? As you notice there are multiple prompts being combined into a super large prompt some of my favourite models are small context windows how can they be improved ? by adding ROPE? or other methods.... so how to really fine tune a model to have new skills... creating the multimode model .or mixture of experts such as mistral and code llama and white rabbit! and phi ?... ? as you notice the super charged models which are proposed for open source are being bought by cloud service providers ! hence creating such model is also need to know before they are no longer available or even finetuned down to unusable models ... ie refusing to answer questions deemed Dangerous by the POWERS AT BE! Or AGAINST FAIR COMPETITION ... ie AI creating Fully working apps !(always not quite right !) ... or not even able to create a transformer ? or GPT model from scratch !... BLOCKED! 4. How to build Relational databases in Pinecone or a vector DB... 4a. how to craft queries which will return relational results .. 4b how to convert / import existing database table into pinecone or a vector DB.. 4c. how to create stored querys which match various question structure which may be asked by the augmented chat system , ie the domain results will be returned from the vector db, and presented as context to the query ie langchain has a table query system ? . Thanks !
This is an excellent in depth tutorial. I just have one question : can we do all these things using Azure OpenAI? How do we integrate that?
I was wondering if is there any way to get the match score in the Langchain version of the RAG output? Retrieving results from Pinecone gives us the core metrics as well, but Langchain documentation does not mention anything. Any insight will be useful.
RLHF. not directly related, but check it, i think you will be interested.
This is nice but do you also have any video with weviate?
Nice video! What is this editor?
would it be possible to use some form of RAG pipeline to help improve an LLM's abliity to classify things that are in a niche domain? eg let's say we have 50 unnusual domains (that weren't in the LLM's training data, or at least, little exposure). We create a vector store where we have 50 labels with a few thousand examples of how each label is expressed. Would that work?
🎯 Key Takeaways for quick navigation: 00:00 🤖 The video will cover building a chatbot using retrieval augmented generation, starting from scratch. 00:26 🧠 The chatbot will use OpenAI's GPT-3.5 model and the LangChain library to answer questions about recent events or internal documentation. 02:43 🤯 Language models like GPT-3.5 can hallucinate answers when they lack knowledge about specific topics not covered in their training data. 07:09 🧩 Chat logs with models like GPT-3.5 can be extended by appending AI-generated responses, creating dynamic conversations. 09:29 💡 Language models like GPT-3.5 rely solely on knowledge acquired during training and have no access to external information. 11:20 🔍 Retrieval Augmented Generation (RAG) adds external knowledge or context to a language model to improve its responses. 14:03 🤔 Language models, like GPT-3.5, may include safeguards to handle questions they can't answer, indicating their limitations. 16:09 📚 Adding source knowledge to the model's input through the prompt can improve its responses to specific queries. 19:33 🚀 Retrieval Augmented Generation (RAG) combines language models with external knowledge sources to enhance responses. 24:25 🔄 When embedding data for knowledge bases, it's essential to batch the process to avoid errors and to consider API limitations. 24:37 📊 Building a fully-fledged Vector database for knowledge retrieval is a crucial step in integrating external knowledge into a language model (LM). 28:30 🧠 Augmenting queries with relevant context from a Vector database enhances the LM's ability to provide informative answers. 30:46 🤖 Language models benefit from augmented prompts with context, allowing them to better answer questions based on the provided information. 33:19 💡 Implementing Retrieval-Augmented Generation (RAG) can significantly improve retrieval performance and answer a wide range of questions accurately. 34:00 💬 When implementing RAG, it's essential to consider token usage, costs, and the balance between external knowledge and LM performance. Made with HARPA AI
What Pinecone has advantages over Chroma or Weaviate? Would everything work in your codebase, say, for Azure OpenAI and Chroma? For data security concerns, most enterprises prefer Azure OpenAI and more secured database choices for embedding. Would be interested to know your expert opinion.
In my corporation, I am thinking of creating a similar chatbot to answer medical questions related to our drugs. We have a lot of products, so I wonder if there are limitations on the context that I can put into the vector database - maybe using some metadata to give the LLMs hints about which documents to search would help.
You can add metadata filters when using RAG with AI agents, or you could maybe try hybrid search for better keyword scoring in your searched
I will start small and test to find the best solution. My bosses are very sceptic towards AI - I must build some case studies to show ROI. Thank you for help.@@jamesbriggs
best of luck!
Great application. I used the OG LLaMa to do this for my Mom actually. She went on an experimental late stage FDA candidate and had some issues when she started treating some side effects. The Drug was new enough that it took a little unpacking to understand more about the chemical make up, but once we got the right info, it immediately caught an interaction with another drug she'd been prescribed to address side effects of the experimental drug. This was overlooked by an entire team of oncologists who were aware of the chemical make up of this drug & still gave her something that caused an interaction. I LOVE this use case! Best of luck!
We are planning to develop a Q&A chatbot for physicians in Canada. I see some challenges related to data preparation for retrieval, as well as monitoring, maintaining, and updating the chatbot performance. The building aspect is the easier part.@@chrismcdannel3908
Great
Thanks!
Is ChatGLM2-6B supported in this colab notebook?
Also, once data has been properly cleaned, validated & prepped to conform to the collection schema after embedding & vectorization it's absolutely imperative that a flush( ) is run on the VDB to prevent decay or loss of data. In the interest of data integrity, and presumable your products performance, it's highly ill-advised to make additions to the data or try to modify it in anyway as this is a known catalyst for data decay and increasingly poor performance as a result.
What does flush do?
What happens if the user later in the conversation asks a task that like tell me more. The rag will fails because it will retrieve "tell me more" content from the indexes, therefore add noisy informations to the chat memory. Chatbot with rag also need some reasoning, so the model decides whether or not to augment the context
yeah you need agents, I made an earlier video on these kzhead.info/sun/e5qbc9WprmuurIE/bejne.html - hope that helps!
what is the process if i what to query chat from cloud mangoDB using llm and RAG
👌🏼
When trying to import pinecone I get errors with the message: If you cannot immediately regenerate your protos, some other possible workarounds are: 1. Downgrade the protobuf package to 3.20.x or lower. 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). I've tried everything I can think of. No idea how to do this.
Hello, I don't get it. What's the difference between the RAG, ConversationalRetrievalChain and RetrievalQA?
What if I have structured data that I want to feed to the RAG. Is that possible?
The major problem I face is, once the embedded data is loaded into the vector db, how do I delete it?. May be I want to delete all data that I fetched from a website and keep everything that I fetched from a SharePoint. How can the embedded data be isolated and deleted ?
updated version datasets==2.10.1 to datasets==2.15.0 to remove the warning embeddings = OpenAIEmbeddings(chunk_size=1) vectorstore = Pinecone( index, embeddings, text_field )
I have a problem, How to make the chatbot don't say "according to your ....." everytime ???
Didn't know Jack Leach was into Machine Learning
I’m not going to play cricket all day every day
The problem is that all the vector databases suck in terms of retrieval
Wait wait. but.. you didn't tell us what to do with this new model now.. so, it's just sitting in Terminal, But, how do we add it to our websites? How to embedd into a chat that is sitting on a wordpress site? Secondly, what if it's being used, and then it answers incorrectly, how to reach it things, or list the items it got wrong to retrain?
with the brain metaphor and the idea of being able to manage someone else's brain and plug in the knowledge needed, you came *this close* to using the Matrix ' I know Kung Fu! ' scene. Which seems pretty much the right metaphor for RAG... kzhead.info/sun/adqGf5eQpICOmGw/bejne.html (15 second snip)
nice haha - matrix kung fu scene will be making an appearance the next time I talk about RAG
5min here 😂
Already haha
You are talking about RAG and in your query you ask "What is special about Llama2". A better example would have been "RAG points to company specific data - say HR corpus". Query then should be company specific "Give me list of company holidays". That Llama2 related query is misleading.
is serverless architecture paid ? i am getting forbidden error
Open AI GPT 3-5 must pay. so can you give solution free?
You could host your own open source LLM (llama2-7b-chat for example) if your hardware supports hosting it.
.
Your videos are great. but it seems like you are repeating your videos in somehow. As I believe that you are a respectable engineer and/or scientist with teaching abilities, I would like to see you are diving deeper into more different subjects. I say that because I am really uncomfortable about the current topic, where all of the people are trying to teach "Creating a Chatbot" for discord coders. I could implement a LLama model just by using your videos, despite that you and I do not share anything in common except Langchain in our codes. However, I spent hours and hours, finally did that. Therefore, I do not think that there is a need for another similar video. To emphasize, I am saying this because I see you as a valuable teacher, and you are not a full-time youtuber. Therefore, if you are spending your free time, I'd like to see different subjects and areas so that I can dive into them with a trustable guide.
yes for sure, the LLM + RAG videos do well so I do talk about them often - but I try and talk about other things too, like deploying open LLMs, guardrails, agents etc - and I will do more in the space of LangChain, hybrid vectors, etc very soon But I'd love to hear what you'd be interested in seeing me talk about?
@@jamesbriggs I already suggested GPU-side of LLMs as I remember. Also, I guess the prompts are wrong templates in your videos, so it would be great if you can take a look at it, because I found the prompt template but could not figure out how to use it in code :)
Could you please make a video about how to use the map reduce and refine Langchain features and show a use case where you grab chunks of data and then use map reduce on them? @@jamesbriggs
long wastful explination, goo directly to the point please.
Speak Fast please. I slept in the middle of video.
KZhead has x1.25 or x1.5 playback speed in bottom toolbar
Would really be awesome to see the same thing implemented without the use of langchain and only relying on pinecone and the openai function calling approach. I’m really trying to get a more native architecture built and don’t want to be constrained to the langchain library if possible. Amazing video as always though James!🥳🦾😎
Oh this is a cool idea actually, will put something together
@jamesbriggs I've been using an implementation that does this for the last 3-4 months (personal project). Feel free to reach out for a collab or even just a mind share.