Agents Lookup

Use cases / Decision models (Jev)

Keep or drop rows of a training dataset by quality

Stream a fine-tuning or web-scraped dataset through Jev with quality questions, keeping clean rows unchanged and dropping the rest.

DifficultyAdvanced
Setup time1 day
CategoryDecision models (Jev)

Author's figure: Author's figure: 24 rows per second measured on one node (Akash Priyadarshi). Reported by the builder, not measured by us.

The use case

ML teams fine-tuning models collect synthetic, scraped or instruction data that is full of junk, and filtering it by hand does not scale. These pipelines stream rows through Jev with typed questions such as whether the answer follows the instruction, whether the text is boilerplate or whether it is on topic. Rows that pass are kept word for word, so nothing is rewritten, and the rest are dropped or set aside. An engineer reviews samples of both kept and dropped rows and checks the effect on an evaluation set before training on the result.

Example workflow

  1. Stream Parquet or JSONL files from Hugging Face Datasets or S3
  2. Ask Jev quality questions per row
  3. Write kept rows and dropped rows to separate files
  4. Review samples from both piles
  5. Train and compare on a held-out evaluation set

Tools

Risks and guardrails

Sources

Written by Agents Lookup. Credits and official pages for the tools this use case names: