Custom RAG assistant trained on company documents
An internal assistant that answers staff questions from the company's own documentation, with citations — instead of generic, wrong answers.
How do I request a refund under the guidelines?
Under Section 2.4, refunds must be requested within 14 days of setup.
"...Critical Priority (P1) issues trigger active response windows within 2 hours. Normal priority..."
The Challenge
Staff wasted time hunting through scattered documentation, and a generic chatbot just made things up. The lack of grounded context meant AI summaries could not be relied upon.
- Internal documentation scattered across Notion, Slack channels, and local PDFs.
- Generic LLM models hallucinating critical operational procedures.
- No baseline evaluation framework to measure if AI updates made answers better or worse.
Operations Bottleneck
Inaccurate answers and database information gaps.
Infrastructure Solved
Prompt templates citing vectors metadata.
The Solution
We built a RAG pipeline over the company's documents — correct chunking, a tuned embedding strategy, and retrieval that returns sources with every answer. An eval set lets the team measure accuracy rather than guess at it.
- Unified ingestion scripts that parse and clean files from Notion and Google Drive.
- Smart chunking with semantic boundaries to preserve sentence and context structure.
- Vector database storage returning exact document paths and page numbers.
Our Approach
Document Audit
Identified and cataloged target document repositories and clean text structures.
Chunking Design
Evaluated semantic sentence chunking algorithms to preserve domain context.
Vector Indexing
Configured Pinecone index with cosine similarity metric and metadata filtering schemas.
Retrieval Tuning
Tuned hybrid search (keyword + vector embeddings) to boost search query match rates.
Chat UI Development
Built a clean Next.js chat layout displaying references alongside answers.
Evaluation Sandbox
Created a testing suite of 50 common questions to benchmark accuracy improvements.
User Journeys
Support Representative
Operations Lead
Tech Stack
Frontend Client
- Next.js (App Router)
- Tailwind CSS
- Framer Motion
AI Orchestration
- LangChain
- OpenAI API
- RAG Pipeline
Vector Search
- Pinecone DB
- Cosine Similarity
- Metadata filters
Evaluations & QA
- Ragas framework
- Grounding filters
Development Process
Semantic Chunking
Avoided generic token-count cutoffs by splitting chunks at logical section boundaries and header levels.
Grounding Guardrails
Programmed prompt layers to instruct LLM models to respond with 'I do not know' if confidence thresholds are missed.
Citation Links
Stored source URLs and page ranges in the vector metadata, enabling frontend UI rendering of direct links.
Accuracy Eval Set
Established regression tests using Ragas metrics to monitor faithfulness and answer relevance.
Results & Impact
Measurable efficiency gains, reduced operations costs, and reliable integrations delivered.
Building something ambitious, or fixing something that's gone sideways?
Tell us where you are and where you're trying to get to. We'll tell you honestly whether — and how — we can help.