CORTEX RAG

CORTEX RAG is an open-source, fully local retrieval-augmented generation (RAG) system built by N Sai Akhil. It runs entirely on your machine using Ollama for LLM inference, requiring no API key, no cloud upload, and no subscription. The project is licensed under MIT and has accumulated over 1,700 GitHub stars since its creation in January 2025.

What It Is

CORTEX RAG is a document question-answering engine that combines nine distinct retrieval and reasoning techniques into a single pipeline. Users upload a PDF, DOCX, TXT, or Markdown file, ask a question in natural language, and receive a cited answer generated entirely on local hardware. The system is designed for both individual developers and enterprise teams that need private, permission-aware document retrieval without sending data to external services.

Nine-Layer Retrieval Pipeline

The core differentiator is the layered pipeline that activates on every query:

• Contextual Retrieval — An LLM prepends situating context to each chunk before indexing, so every vector carries the full document story rather than an isolated fragment.
• RAG-Fusion + RRF — Generates multiple query variants, retrieves independently for each, then merges ranked lists via Reciprocal Rank Fusion for improved recall.
• GraphRAG — Builds a NetworkX knowledge graph over document entities to surface relational context that pure vector search misses.
• Corrective RAG (CRAG) — An LLM grades every retrieved chunk for relevance and silently drops noise before generation.
• Neural Reranking — A Cross-Encoder (ms-marco-MiniLM) reorders retrieval candidates by true query–passage relevance, not just embedding similarity.
• HyDE — Generates a hypothetical answer first to expand sparse queries into a richer dense embedding space before actual retrieval.
• Live Reasoning Panel — Streams the model's chain-of-thought <think> tokens in real time so users can watch the reasoning process.
• Semantic Cache — A cosine-similarity cache at threshold 0.92 returns instant answers for repeated or near-duplicate queries, skipping retrieval and generation entirely.
• Chat Memory — Full multi-turn conversation history flows into every generation call for natural follow-up questions.

Tech Stack

The project is built on a well-defined open-source stack:

Model Flexibility

The sidebar model selector auto-populates from locally installed Ollama models. The README lists tested models including llama3.1:8b (default), qwen2.5:7b for multilingual documents, mistral:7b for long documents, llama3.1:70b for highest quality, and qwen2.5-coder:7b for code and technical documentation. No configuration change is needed when swapping models.

Deployment Options

CORTEX RAG supports three deployment paths: a direct Python/Streamlit run (python -m streamlit run app.py), a Docker Compose setup with Ollama on the host, and a fully containerized Docker option with Ollama running as a separate service. The README also references an enterprise landing page at cortex-rag-beta.vercel.app for organizations that want custom, production-grade RAG systems built on the same nine-layer pipeline.

Current Status

The repository was created in January 2025 and last pushed in June 2026, with the GitHub metadata showing 1,790 stars and 270 forks as of the latest update. The project is actively maintained under the MIT license and explicitly describes itself as free and open-source, with a community support model via GitHub Issues and Pull Requests.