Why this matters
- Four small products teach every moving part: files, sites, structured data, and tools.
- Interviewers ask "tell me about a RAG system you built." A vague chatbot is a weak story. A PDF assistant with an eval set is a strong one.
- Text-to-SQL is the case that shows you know vectors are the wrong tool for a count.
- Shipping one of these is worth more than reading all of them.
Project 1: ask a PDF
The corpus is a file or a folder of files: the employee handbook, a product manual, a contract playbook. The user asks in natural language. You answer from chunks and cite the page.
This is PolicyDesk. It is also the right first project because failure is visible: you can open the PDF and see whether the number was there.
Build outline
- Load with a PDF parser that preserves page numbers.
- Split recursively, with overlap, after stripping headers and footers.
- Embed with one model and store in Qdrant, payload includes page and file name.
- Retrieve top-k and generate with a refusal rule.
- Show the citation in the UI. If you cannot point at a page, you are not done.
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_qdrant import QdrantVectorStore
pages = PyPDFLoader("handbook.pdf").load()
chunks = RecursiveCharacterTextSplitter(
chunk_size=800,
chunk_overlap=120,
).split_documents(pages)
store = QdrantVectorStore.from_documents(
chunks,
OpenAIEmbeddings(model="text-embedding-3-small"),
url="http://qdrant:6333",
collection_name="handbook",
)
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
docs = store.similarity_search("What is the leave policy?", k=4)
context = "\n\n".join(d.page_content for d in docs)
answer = llm.invoke(
"Answer only from this context. If unsure, say you do not know.\n"
f"{context}\n\nQuestion: What is the leave policy?"
).content
Questions to try before you trust it: a number that appears once, a rule that refers to another section, and a question the PDF does not answer. The third must refuse.
Project 2: ask a website
The corpus is documentation or a help center. Users ask "how do I reset the device?" and you answer from the pages you crawled, with the URL as the citation.
What changes from the PDF project
- The loader fetches URLs. You need a list of allowed hosts, not an open crawler.
- HTML must be cleaned. Navigation, "related articles," and footers will otherwise dominate search.
- Metadata includes the URL and the fetch time. Expire pages that 404 on the next crawl.
- Chunk on headings when you can. A help article's H2 is a better boundary than 800 arbitrary characters.
from langchain_community.document_loaders import WebBaseLoader
pages = WebBaseLoader(
["https://docs.example.com/leave", "https://docs.example.com/holidays"]
).load()
# Then the same split, embed, store, retrieve path as the PDF project.
Do not point a loader at a site you do not have the right to copy into an index. For a portfolio, use your own docs or a site that explicitly allows it.
The failure mode unique to the web is boilerplate collapse: every page shares a header, so that header is the nearest neighbor of every query. If your top hit is the cookie banner, your cleaner is the bug.
The rest of this guide is part of Pro
Free covers every fresher guide in full. Pro unlocks Senior, Staff and Architect guides across every track.
Still to come · 4 sections
- Project 3: text to SQL
- Project 4: RAG plus a live tool
- Use cases beyond the exercises
- How to talk about the project
Test yourself
Answer these before moving on — recall is what makes it stick.