The RAG pipeline gives you a complete and end to end system for retrieval augmented generation in iQ Studio. It combines the full process of importing your documents, preparing them for retrieval, and making them searchable. It uses this processed content to return accurate and context aware query responses.
Knowledge bases use the pipeline to improve search relevance and AI generated answer quality. MCP servers rely on it to scale out content processing and support scenarios such as semantic search, document Q&A, and automated content enrichment.
General workflow
- Ingest documents: Submit documents and group them into collections based on your content type.
- Set up configuration: Select the embedding model, reranker, and chunking strategy for each collection.
- Process content:
- The pipeline validates and parses each file.
- It applies the appropriate chunking template such as manuals, emails, or books.
- It divides the content into chunks and adds metadata.
- Generate embeddings and store them:
- Each chunk is converted into a vector embedding.
- Embeddings and metadata are stored in Qdrant and organized by collection.
- Retrieve and augment answers:
- When you query a knowledge base or MCP server, the pipeline finds the most relevant chunks using similarity search.
- These chunks are used to build accurate and context aware responses.