# PixelRAG

By **startrail-org** · Software & Tools

Visual RAG pipeline that retrieves over screenshot tiles instead of parsed text, preserving tables, charts, and layout for more accurate document search.

- Source: https://github.com/StarTrail-org/PixelRAG
- Repository: https://github.com/StarTrail-org/PixelRAG
- Install: `pip install pixelrag # Optional pipeline extras pip install 'pixelrag[embed]' pip install 'pixelrag[index]' pip install 'pixelrag[serve]'`
- Tags: rag, vision, open-source, cli, python, faiss, embeddings, agent
- Pricing: free
- Upvotes: 1

## Features

- Renders web pages, PDFs, and images to screenshot tiles via headless Chromium (Playwright/CDP)
- Visual and text query search against a FAISS index of screenshot embeddings
- Hosted endpoint with 8.28M Wikipedia pages pre-indexed, no API key required
- Hybrid text-plus-image query support for multimodal retrieval
- Claude Code plugin (pixelbrowse) for in-session visual page browsing
- Modular pipeline: install only the stages (render, embed, index, serve) you need
- LoRA-fine-tuned Qwen3-VL-Embedding-2B model with published adapters on Hugging Face
- Self-hostable FAISS search API via FastAPI, CPU or GPU

---

## Why it matters
Text-based RAG silently drops tables, charts, and layout when it parses HTML to plain text. PixelRAG skips parsing entirely: it renders every document as screenshot tiles and retrieves over the images, so a reader model sees the data exactly as it appears on the page.

## The big picture
The pipeline comes from Berkeley SkyLab, BAIR, and Berkeley NLP. Its core claim, per the README title, is that "web screenshots beat text for retrieval-augmented generation." The key insight: visual structure carries meaning that no parser preserves.

## How it works
Two components do the work. First, `pixelshot` renders any URL or PDF into tiled screenshots via headless Chromium (Playwright/CDP). Second, a `Qwen3-VL-Embedding-2B` model, LoRA-fine-tuned on screenshot data and published at `Chrisyichuan/wiki-screenshot-embedding-lora`, embeds those tiles into a FAISS index where visual content is directly retrievable.

The pipeline stages are modular: render, chunk, embed, build-index, and serve each install independently. The hosted `api.pixelrag.ai` endpoint ships a pre-built index of 8.28M Wikipedia articles, free and keyless. Downloading the base FAISS index locally clocks in at roughly 217 GB.

## Zoom in
The `pixelbrowse` Claude Code plugin wires `pixelshot` directly into Claude Code sessions. No MCP server, no backend: Claude calls `pixelshot` locally and reads the resulting image, so it sees charts and tables the same way a person does. Install with `claude plugin install pixelbrowse@pixelrag-plugins` after putting `pixelshot` on your PATH via `uv tool install pixelrag`.

The search API also accepts image queries and hybrid text-plus-image queries, making it usable for visual-similarity search, not just text-to-screenshot retrieval.

## Yes, but
Indexing your own documents requires Python 3.12+ and a GPU (or Apple Silicon) for reasonable speed, the README cites roughly 3 minutes on Apple M-series and 1 minute on GPU for a single PDF. The base Wikipedia FAISS index is around 217 GB, so self-hosting it demands significant disk. The bundled headless Chrome only auto-installs on linux-x64; macOS and Windows need a system Chrome or Playwright's Chromium.

## The bottom line
PixelRAG is the right reach when your documents are visually rich and text parsers fail you. The hosted Wikipedia endpoint lets you validate the approach in minutes, and the modular CLI means you can swap in only the stages your project actually needs.

## FAQ

### What is PixelRAG and how does it differ from standard RAG?

PixelRAG is a visual retrieval-augmented generation pipeline that renders documents as screenshot tiles and retrieves over those images, rather than parsing text. Standard RAG pipelines parse HTML or PDF to plain text, discarding tables, charts, and visual layout in the process. PixelRAG keeps that structure intact by treating every page as an image, then using a vision-language embedding model to match queries against the visual content. It was developed at Berkeley SkyLab, BAIR, and Berkeley NLP.

### How do I install and run PixelRAG?

Install the base package with `pip install pixelrag` (requires Python 3.12+), which gives you the `pixelshot` CLI for rendering pages to tiles. For a full local pipeline, install the index extra with `pip install 'pixelrag[index]'`, write a `pixelrag.yaml` config pointing at your documents, run `pixelrag index build`, then serve with `pixelrag serve`. You can also query the hosted Wikipedia index immediately, no setup or API key needed, via `curl -X POST https://api.pixelrag.ai/search`.

### Is PixelRAG free and open source?

Yes, PixelRAG is released under the Apache-2.0 license and the full codebase is on GitHub. The hosted `api.pixelrag.ai` endpoint serving 8.28M Wikipedia pages is also free and requires no API key. Self-hosting your own index requires your own hardware and roughly 217 GB of disk for the base Wikipedia FAISS index.

### What is PixelRAG best for?

PixelRAG is strongest when your documents are visually dense: pages with tables, charts, infographics, or layouts that text parsers would mangle. It is also a good fit for giving Claude Code visual browsing capability via the `pixelbrowse` plugin, letting Claude see charts and diagrams the way a person does instead of reading raw HTML. The hosted Wikipedia API makes it easy to validate the approach on a real-world scale before committing to a self-hosted index.

### How does PixelRAG compare to text-based RAG pipelines?

Text-based RAG converts documents to plain text chunks, which silently drops tables, charts, and layout; PixelRAG renders to screenshot tiles and retrieves the images directly, preserving that visual structure. The tradeoff is infrastructure: PixelRAG requires a vision-language embedding model (Qwen3-VL-Embedding-2B), a FAISS index, and a rendering step, whereas text RAG pipelines are lighter to self-host. Per the README's research title, the Berkeley team found that screenshot-based retrieval beats text retrieval for web documents.

### What are the main limitations or requirements of PixelRAG?

PixelRAG requires Python 3.12+ and works best with a GPU or Apple Silicon for embedding at reasonable speed, the README notes roughly 3 minutes per PDF on Apple M-series and 1 minute on GPU. The bundled turbo headless Chrome auto-installs only on linux-x64; macOS and Windows users need a system Chrome or Playwright's Chromium already installed. Downloading the base Wikipedia FAISS index locally is approximately 217 GB, making full self-hosting storage-intensive. The training environment pins specific versions (torch 2.9.1, transformers 4.57.1) in a separate project, though you do not need to retrain to use the published adapters.

---
[View on Analog](https://analoghq.ai/pt-br/startrail-org/software-tools/pixelrag)