Vector RAG, Jev + RAG and GraphRAG: Three Fixes for Three Different Retrieval Failures

Those failures map to three different fixes. Vector RAG supplies missing knowledge for local questions. GraphRAG supplies structure for corpus-wide or relationship-heavy questions. Jev + RAG (TypeSafe AI’s Jev as a typed decision step after retrieval) filters and routes candidates so only passages that answer — or explicitly conflict — reach the generator, and so the application can abstain. None replaces retrieval. This article is a decision frame for architecture teams — not a product ranking, and not a restatement of production GenAI evaluation and failure modes.

Three approaches in production terms

Vector RAG combines a generator with a dense vector index: retrieve by similarity, generate from the shortlist, and update knowledge by changing the index rather than retraining the model (Lewis et al., 2020). Jev + RAG keeps that retrieval path and inserts a typed judgment step — relevance, evidence, conflict, optional sufficiency — before generation (TypeSafe RAG cookbook). GraphRAG builds an entity–relationship graph (often with LLM extraction), organises it into communities with summaries, and answers global questions by map-reduce over those summaries (Edge et al., 2024).

Vector RAG — the baseline and its two blind spots

Lewis et al. defined retrieval-augmented generation as parametric memory (the generator) plus non-parametric memory (a dense index accessed by a neural retriever). The operational gift is freshness: swap or update the index without retraining. Approximate nearest-neighbour search is the usual query path — fast, and not guaranteed to be exact.

Edge et al. call this family vector RAG and locate its strength clearly: it works well when the answer lives in a small set of records similar to the query. Microsoft Research’s LazyGraphRAG write-up makes the same point from the other direction — vector RAG “excels for local queries.”

Two blind spots keep recurring.

Blind spot 1 — aggregation. Corpus-wide questions (“what are the main themes across this estate?”) give similarity search almost nothing to steer on. GraphRAG’s documentation states the same limitation for baseline RAG when answers require aggregating information across the dataset.

Blind spot 2 — “retrieve more” is not a fix. Liu et al. (Lost in the Middle) show accuracy degrades when relevant material sits in the middle of long contexts, and that reader accuracy saturates long before retriever recall improves. They recommend reranking and truncating the ranked list — not stuffing the prompt. Cross-encoder re-ranking is established prior art. Cache-augmented and long-context alternatives are a separate design question (companion CAG vs RAG Insight); they do not erase positional degradation.

Vector RAG remains the most mature baseline — peer-reviewed since 2020, deep managed-vector ecosystem — and the right default when questions are local and the corpus changes often enough that index hot-swap matters.

Jev + RAG — a decision step between retrieval and generation

What Jev is (and is not)

Jev is TypeSafe AI’s flagship model and its first “System One” model, announced 15 September 2026 in early access. Given a state (text or structured fields) and typed questions, it returns structured answers with probabilities — Choice, Score, or Noul — and does not generate text (TypeSafe introduction; System One concepts). TypeSafe states it is not a chat or code-completion LLM and not a drop-in replacement for the model behind coding agents.

As of 8 October 2026, TypeSafe’s models page lists jev-1.13.0 (aliases jev-latest / jev-preview point at the same build): text-only input; 64k tokens per request, with 32k for state plus the longest question. Rate limits are adjusting dynamically. Pin a versioned ID once thresholds are tuned — aliases move when releases ship. Pricing is on the vendor page; early-access rates change, so they are not hardcoded here.

The vendor launch post says Jev “can’t hallucinate.” In vendor terms that means outputs stay inside the declared answer types — it does not emit free text outside the schema. TypeSafe’s own docs add that calibration does not guarantee that an individual answer is correct. A third-party preprint (Porcedda, arXiv 2609.35342) argues the calibration claim was not backed by a public test at the time of writing and reports Choice probabilities as less well calibrated than Noul or Score on a synthetic set. Treat “can’t hallucinate” as a type-safety claim, not a correctness claim, and treat calibration as a vendor claim with disputed third-party evidence.

The pattern: similarity is not answerability

Ranking and answerability are different questions. A passage can be about the topic without answering it. TypeSafe’s classifying-passages cookbook places one Jev request per retrieved passage with four Noul questions — relevant? contains answer evidence? contradicts the query’s premise? contains prompt injection? — then routes in application code to evidence, conflict, or drop. Conflicting evidence stays in its own block rather than being silently discarded. An optional sufficiency check lets the app return “not in the documents” instead of calling the generator. An LLM still writes the answer when evidence survives.

Spring AI exposes the same idea as Modular RAG post-processors — Jev sits after retrieval, with access to the user’s question. Third-party explainers agree: Jev does not replace the embedding model, vector store, or generator; retrieval still sets the cap.

The idea is not new. Corrective RAG uses a lightweight retrieval evaluator; Self-RAG critiques passages with reflection tokens. Jev is a hosted, typed implementation of that gating pattern — useful when you want probabilities, thresholds in code, and an audit log of per-passage decisions.

Limits to state plainly

• Cannot recover excluded evidence. If the retriever never fetched the answer, a gate can only abstain. Samelogic’s small synthetic study illustrates a BM25 shortlist that dropped the relevant record; Jev then refused an answerable question.
• Not a corpus-scale retriever. Request bounds (64k / 32k; Choice option caps) make “put the whole corpus in one Jev call” a bounded technique, not an index substitute.
• Not a security boundary. TypeSafe states the injection question is a filter; the generator must still treat every passage as untrusted.
• Documented failure modes for Jev 1.13 include literal reading, arithmetic and date comparison, multi-hop indirection, large state with irrelevant detail (“context rot”), and Choice option-order bias (jaggedness notes).
• Cost and latency scale with candidates. The cookbook sends one request per passage. A third-party self-published benchmark (emretheus, September 2026) reported multi-second p50 rerank latency versus hundreds of milliseconds for an NVIDIA cross-encoder on the same frozen top-20 — a latency comparison, not a leaderboard.
• Evidence maturity is early. As of October 2026, public Jev evidence is vendor docs plus early third-party studies and preprints — no peer-reviewed evaluation found. Do not treat cookbook or blog scores as settled benchmarks.

GraphRAG — structure for global and relational questions

Microsoft’s GraphRAG pipeline uses an LLM to extract an entity–relationship graph from source text, partitions it with hierarchical Leiden community detection, pre-generates community summaries, and answers global questions by map-reduce over those summaries. The query engine offers local search (entities plus linked text chunks — entity-specific questions), global search (map-reduce over community reports — documented as resource-intensive), DRIFT search, and a basic vector search for comparison (GraphRAG query overview).

On two roughly million-token corpora, Edge et al. report GraphRAG global approaches beating vector RAG on comprehensiveness (about 72–83% and 72–80% win rates) and diversity, judged by an LLM, while vector RAG produced the most direct answers. Those figures are preprint, author-reported, LLM-as-judge, global sensemaking only, two corpora. The same paper’s indexing example took 281 minutes on a podcast corpus; root-level summaries needed more than 97% fewer context tokens than map-reduce over source text. The open-source README warns that indexing can be expensive and to start small; the repository presents itself as a demonstration, not an officially supported Microsoft product.

LazyGraphRAG (Microsoft Research) defers LLM work to query time and reports indexing cost identical to vector RAG and about 0.1% of full GraphRAG on Microsoft’s own evaluation — vendor-research, not an independent audit. Amazon Bedrock Knowledge Bases GraphRAG with Neptune Analytics is a managed hybrid: vector search first, then graph traversal. As of 8 October 2026, AWS documents these limits: no custom graph-build configuration; no autoscaling for Neptune Analytics graphs; Amazon S3 only; up to 1,000 files per data source (raisable to 10,000 on request). Region availability is listed in the same AWS guide — re-check before design commitments.

Peer-reviewed caution matters. Chen et al. (ACL GEM 2026) ask “Is GraphRAG needed?” on STaRK-Prime and answer contextually: GraphRAG helps complex relational queries; simpler RAG variants can compete with lower overhead; a KG computed from documents underperformed a predefined one; pure graph search without semantic signals performed poorly; cheap 1-hop neighbour augmentation improved basic RAG; and retrieval metrics can overstate benefit when the generator under-uses expanded context. Single dataset, single LLM — still the strongest public counterweight to “always add a graph.”

Freshness is an extra pipeline step (update CLI methods; incremental proposals such as LightRAG). Graph quality is the silent failure mode — a bad extracted graph can underperform basic RAG.

Decision frame — which failure are you fixing?

Ask these before branding the stack:

1. Is the failure recall (missing knowledge), precision / answerability (wrong neighbourhood or unsupported passages), or structure (relationships and corpus-wide themes)?
2. Are production questions mostly local, or do they require global / multi-hop reasoning?
3. How often does the corpus change, and who owns re-index or update cost?
4. Who must audit why a passage was kept, dropped, or marked as conflict?
5. Can you amortise graph indexing over many global queries — or is a cheaper 1-hop augmentation enough?

Combinations are the norm, not the exception. Bedrock GraphRAG starts with vector search. GraphRAG local search mixes graph structure with text chunks. LazyGraphRAG blends best-first and breadth-first retrieval. A typed decision step can follow any retriever in principle — including graph retrieval — though public sources demonstrate Jev mainly after BM25 or embedding retrieval.

Evaluate before you adopt

Separate retrieval metrics from answer quality. The GEM 2026 retrieval–generation gap and Lost in the Middle both warn that better shortlists do not automatically become better answers. Include unanswerable and multi-hop questions in evaluation sets. Tune abstention thresholds separately from ranking thresholds — vendor cookbooks call their cut-offs starting points, not defaults. Pin hosted decision-model versions and re-check request limits before go-live. Pipeline staging, observability, and permission boundaries belong in a companion Insight on production RAG pipelines; evaluation acceptance and graceful failure sit in the published production GenAI piece.

FAQ

Does Jev replace RAG?
No. Every primary and third-party source reviewed positions Jev as a decision component beside retrieval. An LLM still writes the answer; retrieval still sets the ceiling on available evidence.

What does “can’t hallucinate” mean for Jev?
Only that outputs conform to the declared answer type. It does not mean the judgment is correct. Vendor docs separate calibration from per-answer correctness; a third-party preprint disputes the public calibration evidence.

Is GraphRAG always better than vector RAG?
No. Peer-reviewed work finds it helps for complex relational queries and is not always needed; computed graphs can underperform; cheap 1-hop augmentation sometimes closes much of the gap.

Can these be combined?
Yes. Managed GraphRAG designs often start from vectors; a decision step can sit after any retriever. Choose by failure mode, then compose.

What this is not

• Not a vendor bake-off or endorsement of TypeSafe AI, Microsoft GraphRAG, or Amazon Bedrock.
• Not a claim that Yottalyte has deployed, benchmarked, or partnered on Jev or GraphRAG.
• Not a pipeline-ops manual (chunking, embedding ops, evaluation harnesses) — that is the companion RAG pipeline Insight.
• Not a long-context / cache-augmented generation comparison — that is the companion CAG vs RAG Insight.
• Not a restatement of Moving Generative AI from Demonstration to Production.
• Not formal partnership or certification language — technology names here are capability coverage.

Operational implications

If the team only argues acronyms, the wrong failure stays unfixed. The durable deliverable is an architecture choice tied to query shape and failure mode: vector RAG for local freshness, a typed decision step when answerability and abstention matter, graph indexing when structure and global sensemaking justify the cost — usually in combination, with evaluation that separates retrieval from generation.

That work sits under AI & Intelligent Systems and Data Engineering & Modern Data Platforms. For white-label and co-delivery, partners retain client and commercial ownership; implementation capacity owns the artefacts that decide the pattern: query-shape inventory, abstention policy, graph re-index ownership, and version pins for any hosted decision model. Prefer durable retrieval architecture over impressive demos.

References

Primary sources used for technical claims in this article. Inline citations remain authoritative for the locus of each claim.

1. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis et al., NeurIPS 2020
2. Lost in the Middle: How Language Models Use Long Contexts — Liu et al., TACL
3. From Local to Global: A GraphRAG Approach to Query-Focused Summarization — Edge et al., Microsoft Research (preprint)
4. GraphRAG query overview — Microsoft
5. microsoft/graphrag — Microsoft (open-source README)
6. LazyGraphRAG: Setting a new standard for quality and cost — Microsoft Research
7. Build a knowledge base with Amazon Neptune Analytics graphs — AWS Bedrock
8. Is GraphRAG Needed? From Basic RAG to Graph-/Agentic Solutions with Context Optimization — Chen et al., ACL GEM 2026
9. Graph Retrieval-Augmented Generation: A Survey — Peng et al., ACM TOIS (open version: arXiv 2408.08921)
10. Introducing System One Models & Jev — TypeSafe AI (vendor)
11. TypeSafe docs: Introduction · System One · Models · Classifying RAG passages · Jev 1.13 jaggedness — TypeSafe AI (vendor)
12. Spring AI Modular RAG and TypeSafe Jev — Spring blog
13. Corrective Retrieval Augmented Generation — Yan et al. (preprint)
14. Self-RAG — Asai et al. (preprint)
15. Jev thinks "I don't know", but doesn't say it: Sys1Cal-v1 — Porcedda (preprint)