Live code review: Pinecone semantic search
In this live code review, Zack and Roie build a semantic search engine with Typescript. Throughout the session, we'll explore some of the basic concepts behind semantic search, and take a deeper dive into the code base of the semantic search Typescript example available in Pinecone's learning center.
Timestamps - so you can jump to what’s most interesting to you:
0:11 - Intro
1:05 - Basic concepts of lexical search
4:55 - What are embeddings?
9:08 - How does semantic search work?
15:30 - Comparing semantic and lexical search
19:40 - Build session intro, CSV loader
26:40 - CSV loader done, short review of utilities
28:00 - Introducing Transformers.js
32:50 - Short look at why types can be helpful
35:40 - Embedding in batches
39:00 - Ideas around using workers to parallelize embeddings
40:35 - A client helper function
44:10 - The main portion of the indexing process
55:40 - Where to find the repo
56:26 - Querying the index
This is such great stuff, especially the explanations/comparisons between lexical/semantic search and how neural networks work. If I wanted to introduce embeddings to someone, this would be the video I'd use. Thanks and keep them coming!
I joined at the end! Great presentation! How deep can an index be starting with the vector? Can it have image file name, date created, image size etc?