Rerank and filter retrieved passages before an LLM answers
After keyword or vector search, have Jev score each candidate passage for relevance and drop weak ones so the writer only sees good context.
Author's figure: Author's figure: 20 candidates scored per request with a 0.6 passage threshold (NO1ennn). Reported by the builder, not measured by us.
The use case
Teams running internal search or RAG chatbots often feed the answer model passages that look similar but do not answer the question, which leads to confident wrong answers. In these builds full-text or vector search pulls the top candidates, Jev scores them all against the question in one request, and only passages above a threshold reach the writer. One design adds gates for whether the question is answerable at all and whether a passage contains injected instructions, then tests every sentence of the answer against the passage it came from. If nothing passes, the system says it does not know. Owners review unanswered questions to find gaps in the documents.
Example workflow
- Retrieve the top 20 candidates with Postgres full-text or Vectorize
- Ask Jev a Score question per passage in one request
- Pass only passages above a threshold to the answer LLM
- Check each claim in the answer against its source passage
- Log unanswered questions for the docs owner
Tools
- Jev (TypeSafe AI)
- Cloudflare Vectorize
- LangChain
- PostgreSQL
Risks and guardrails
- A strict threshold returns no answer too often
- Passages that are relevant but long may score low
- Retrieval misses cannot be fixed by reranking
Sources
Written by Agents Lookup. Credits and official pages for the tools this use case names:
- A RAG loop where Jev decides what is real by NO1ennn on Made with Jev: madewithjev.com/builds/stanford-rag-jev-loop
- A RAG loop where Jev decides what is real: original post or repo by NO1ennn: x.com/N01ennn/status/2105606036639932433
- ensk by Jökull Sólberg Auðunsson on Made with Jev: madewithjev.com/builds/ensk
- ensk: original post or repo by Jökull Sólberg Auðunsson: github.com/jokull/ensk
- jev-ranker by Likitd_ on Made with Jev: madewithjev.com/builds/jev-ranker-rag
- jev-ranker: original post or repo by Likitd_: x.com/Likitd_/status/2102983213455175740
- JevRelevanceRetriever by SakshamMalhot27 on Made with Jev: madewithjev.com/builds/jev-relevance-retriever
- JevRelevanceRetriever: original post or repo by SakshamMalhot27: x.com/SakshamMalhot27/status/2102739222574821827
- Jev for RAG by Viv on Made with Jev: madewithjev.com/builds/jev-for-rag
- Jev for RAG: original post or repo by Viv: x.com/Vtrivedy10/status/2102808794493321714
- LangChain (official page): www.langchain.com (checked 2026-11-19)