Agents Lookup

Use cases / Decision models (Jev)

Check extracted document fields against the source text

After an LLM extracts fields from a document, Jev checks whether the source text actually supports each value.

DifficultyIntermediate
Setup time3-6 hours
CategoryDecision models (Jev)

Author's figure: Author reports the LLM extractor invented an unsupported value in 6 of 17 runs, and Jev scored the planted value at 0.02; 10 fields checked in about 0.9 s.. Reported by the builder, not measured by us.

The use case

Teams that use LLMs to pull fields from loan forms, claims or invoices sometimes get values the document never states. Jev cannot extract values, but it can answer whether the OCR text supports a given extracted value, with a probability. Fields with low support are flagged before they enter the system of record. The author also found Jev treats fax banners as content and cannot read checkbox layout, so those need cleanup or a person. An operator reviews every flagged field, and anything financial or legal is signed off by a person.

Example workflow

  1. OCR the document with LLMWhisperer or another OCR tool
  2. Extract fields with your existing LLM extractor
  3. Ask Jev one yes/no question per field: does the text support this value?
  4. Flag fields below a set probability
  5. Operator checks flagged fields against the original before saving

Tools

Risks and guardrails

Sources

Written by Agents Lookup. Credits and official pages for the tools this use case names: