Extract Topics From Video/Audio With LLMs (Topic Modeling w/ LangChain)
2024 ж. 13 Мам.
14 080 Рет қаралды
Learn To Build With AI: mail.gregkamradt.com/signup
Twitter: / gregkamradt
Code: github.com/gkamradt/langchain...
0:00 - Intro
1:13 - Code Intro
2:46 - LLMs and Set Up
4:43 - Extracting Topic Titles
8:29 - Creating Structured Data
9:47 - Expanding on Topics
15:00 - Bonus: Add timestamps to topics
Thank you for the thorough introduction and sharing of your research.
I love your videos, just subbed. I've been watching your videos for a few months now and all of your content is gold. I've been in tech for ages and writing python like about 8 years, and I'm just so blown away by how much power LLM's have put into our hands with a simple API call. I'm using GPT all over the place in my personal tools right now, which is in turn helping me write even more, better code. I've been doing a hobby project where I have GPT doing virtual Tarot Readings in Streamlit app and with just a few nights and weekends of work it is capable of things that, without an LLM I doubt I could have EVER made happen in any amount of time. I haven't started using langchain yet because I've wanted to work directly with the prompts and responses for a while to get a good feel for how the models work, but I can already tell it's an amazing framework. Thanks for making this content!
Awesome! I love it - thank you for sharing your thoughts and comments! Make sure to share what you build, I'd love to check it out
This is gem. Ty. Very well structed topic extraction.
Awesome - thank you! What other domains do you think it’d be good for?
Thank you for an awsome walk thru video very much. Every bits are informtive. 😊
Lovely - thank you!
Such a good video. You do an incredible job of balancing the technical breakdown, the code itself and how it relates to the real world. Nice job!
Awesome love it - thanks Ed! I might take this one a bit further but we'll see
Great lectures.
This is great content, thank you so much! I learn so much from you every time I watch your videos
Awesome! Thank you Maya. I enjoyed your video on the one-prompt experiment a few days ago. Is that your editing? Or do you use an editor?
@@DataIndependent Thank you so much, I'm really glad you enjoyed it ☺ yep, just me and free Canva/Davinci Resolve accounts. And a looot of hours 😅
Awesome - I'm still on the Davinci resolve game myself
Helpful
Very helpful. Thank you!
Awesome! Thank you - What do you want to see more of?
Nice! I like this🦾 Tku for sharing👏🏼😎
Awesome - what domain do you think it would be good for?
Thank you for the great information! I hope you’ll consider doing a tutorial on how to use a GUI like flowise ai with langchain.
Thank you💗
Totally! What domain do you think extracting more topics or structured data from would do well?
Apt explanation! Thanks
Awesome, thank you
That's amazing, great walkthrough, very easy to follow. I guess one improvement would be to look at the final timestamps, and remove any that are within say 30 seconds of the previous topic. At 12:32 in the example timestamps you have two topics. I guess you could either do this manually, or in Python. Perhaps given that it might only take 30 seconds to do so perhaps human tweaking right at the end is the way to go there.
Ah yes! Great call that actually slipped by me. Ya I think first pass is a human goes through and does this one off for quick fix. My guess is these two topics happened via a monologue and there was only a single time stamp available. I pulled this from a source that had a timestamp for each word so we could likely break this down even further for better resolution
I deleted my last comment because i was wrong and i feel like a moron. Anyways, these videos are invaluable and I hope you keep posting. Not only is the content cutting edge and already in high demand, but your style of presentation/explanation is top notch.
Nice! Thanks Ali - what was the previous comment? Was it negative? I’d love to hear any feedback
@@DataIndependent nothing negative! my official feedback is that you're fantastic
I am a new in this channel, I will start with you but I have a question, Is playlist complete? and will explanation LangChain? thank you very much. and another question sir Are there any requirements to understand this content other than Python?
Hi Greg. Thanks for your amazing video😃 What I want to know is that there are several methods to convert context into structured data using LLM, such as the Schema you used in the video or the Pydantic method provided in the LangChain official documentation. Now I would like to ask, in terms of output stability, which method would you recommend? Because for me, using Schema to define the output of LLM seems more intuitive and straightforward than Pydantic. Thank you for your response.
Awesome thanks! I haven’t seen data that suggests one is better than the other yet. So we this point my only recommendation is to do the one you’re comfortable with
@@DataIndependent Thank you for your response. I have an additional question that I would like to seek your advice on.. I currently have a task to analyze 2000 ~ 3000 hotel reviews (CSV file) by LangChain template. The goal is to summarize each review and classify the level of public relations handling priority for that review. I have already successfully obtained the desired summaries and priority classifications from 30 hotel reviews. However, I am unsure how to scale up the process. Due to the large number of reviews (2000 ~ 3000), there will inevitably be an issue with excessive tokens. Since my task is to process each individual review, I am concerned that using a text splitter may result in the loss of some reviews or that chunk overlap may cause duplication in the processing of my reviews. I would like to ask if you have any suggestions regarding data segmentation. Should I first perform the segmentation locally and then use a method to read multiple files to be processed by LLM? Thank you for your patient response. Big Fan from Taiwan
Great content. Wouldn't abbreviating the names in the transcript help reducing the token size or is it the same? Likewise converting the timestamps to base64..?
You could abbreviate the names, but the savings wouldn't be too high imo. Same with timestamps, a few tokens but not a material amount
Hi, Thanks for this. Is it possible to make a tutorial on product recommendation through LangChain & OpenAI?
Hey, thank! What types of recommendations? What's the use case?
@@DataIndependent Hi, The use case can be product Recommendation on an e-commerce platform which can be broadly of two types 1. Similar products - I have selected nikey shoes then show products similar to nikey shoes. 2. Bundling - I have selected nikey shoes then show products that can complement shoes. One approach that I have in mind is to convert the entire product page content into embeddings and then run retrieval qa chain. What are your thoughts on that? Really appreciate it.
Awesome video, thanks for sharing! ✨ Are there any free alternatives to openai models? ... For example, how to rewrite the same code if there are equivalent models offered by Huggingface?
Try this out python.langchain.com/docs/integrations/llms/huggingface_hub
@@DataIndependent Thank you! 😊
Great video, thanks! I ran into a problem this afternoon with the reference code. I can't import Pinecone. Any suggestion? Thanks!
Have you installed it?
@@DataIndependent Yes, I tried but failed. I am using win10. Could you pls tell the right way of installing it? Thanks!
how do you obtain the GPT-4 API? is it not given to only a select few people? Btw, love your content!
Awesome thank you! I signed up on the waitlist and got it a little while ago
can you make a video on how to upload image as well as pdf and text prompts in LangChain ?
Yes! Great video idea
How much does it cost to run this model on OpenAI?
First off, thanks for the subscribe! For the costs I have to give a lame answer...it depends! On what model you use and how long your transcript is
Hi Greg, for your transcript txt files, did you manually copy/paste them from youtube? Or did you use something like Whisper to generate them? I don't believe the youtube api provides timestamps when fetching transcripts...
For this tutorial I used www.assemblyai.com/ and then manually added readable names to their speaker labeled
@@DataIndependent just as a followup, we did find that the youtube api does provide timestamps in json format
hi @@calebsuh , can you help me how to get the transcript alongwith timestamp. I am using this to load the youtube transcript . from langchain.document_loaders import KZheadLoader def create_db_from_youtube_video_url(video_url: str) -> FAISS: #loads youtube transcripts from the given url loader = KZheadLoader.from_youtube_url(video_url) transcript = loader.load() print(transcript[:300]) This prints the document of transcipt but timestamp is not there. Can you help me please?
Do you know if I can get structured data using a gemini model? Seems like langchain only supports interfacing with OpenAI for this specific approach. Not sure if I am missing something
I haven't seen function calling with gemini yet, but you can try to work with instructor (the library) for this
@@DataIndependent Thanks, will do!
When I run this on longer text I get time out errors with gpt-4-0613 but no problems with gpt-3.5-turbo-0613 when I call load_summarize_chain. Is there some maximum time limit?
The timeout errors is because gpt4 is taking too long. You can increase the timeout when you make your LLM or you can make your prompt output smaller which should help
sorry in this exercise how did you get the transcription with timestamp in the first place?
This was made via deepgram and took some massaging to gather
I finished your langchain series, isn't there a continuation?
What do you want to see next?
@@DataIndependent First, thank you for the reply, I'm subscribing rn. If I may request, I want you to build something that is unprecedented yet practical and of course can be monetized. Most people build similar products, they are competing on highly saturated market. I'm just nobody who has crazy ideas from non computer science field who just jumped to this AI thing. And thankyou for sharing your knowledge.
Also, since I'm from business field who (I think) could see something, and you are one of the people who can actually build it (this is my 2nd month of learning AI, ML even CS haha) Build something around "Safety, Privacy, Ethical & Protection" for untapped B2B sectors. That's where the money is, field that is not so red and people who wanna buy (B2B). That's what I'm heading to.
I am getting an error after I execute this line "topics_found = chain.run({"input_documents": docs})". Error: Traceback (most recent call last) Input In [9], in () ----> 1 topics_found = chain.run({"input_documents": docs}) File ~/opt/anaconda3/lib/python3.9/site-packages/langchain/chains/base.py:290, in Chain.run(self, callbacks, tags, *args, **kwargs) 288 if len(args) != 1: 289 raise ValueError("`run` supports only one positional argument.") --> 290 return self(args[0], callbacks=callbacks, tags=tags)[_output_key] any idea? It is very strange
That does looks weird. Have you updated langchain recently?
@@DataIndependent thats what I thought so I re-install it but still get this error. I am using the latest langchain 0.0.227 with Python 3.9.12. Thanks for responding
I’m not sure. I’ll check my code and make sure it runs for me but unsure. That’s an annoying bug
For something like timestamp being off by one character could give vastly different and unintended results. I wonder if there would be a way where you could force the formatting and then do a search within the transcript to double check and make sure the timestamp selected is in the original transcript. Have you looked into Microsoft's Guidance? That could force the formatting.
Like a hallucination of a timestamp? I could see that being a problem but I'd want to run it through benchmarks to see. You could regex out all the time stamps and do a quick look up with what the LLM returned to double check. No LLM required for that step