Agents Lookup

Use cases / Decision models (Jev)

Rerank and filter retrieved passages before an LLM answers

After keyword or vector search, have Jev score each candidate passage for relevance and drop weak ones so the writer only sees good context.

DifficultyIntermediate
Setup time3-6 hours
CategoryDecision models (Jev)

Author's figure: Author's figure: 20 candidates scored per request with a 0.6 passage threshold (NO1ennn). Reported by the builder, not measured by us.

The use case

Teams running internal search or RAG chatbots often feed the answer model passages that look similar but do not answer the question, which leads to confident wrong answers. In these builds full-text or vector search pulls the top candidates, Jev scores them all against the question in one request, and only passages above a threshold reach the writer. One design adds gates for whether the question is answerable at all and whether a passage contains injected instructions, then tests every sentence of the answer against the passage it came from. If nothing passes, the system says it does not know. Owners review unanswered questions to find gaps in the documents.

Example workflow

  1. Retrieve the top 20 candidates with Postgres full-text or Vectorize
  2. Ask Jev a Score question per passage in one request
  3. Pass only passages above a threshold to the answer LLM
  4. Check each claim in the answer against its source passage
  5. Log unanswered questions for the docs owner

Tools

Risks and guardrails

Sources

Written by Agents Lookup. Credits and official pages for the tools this use case names: