Score support calls against a coaching scorecard
Jev answers yes/no and rubric questions about each agent turn in a call transcript, and code builds a scorecard a coach reviews.
Author's figure: README reports scoring 9 seeded calls with Jev for about $0.007 at a median 200 ms per request, versus about $0.021 for Gemini Decisions.. Reported by the builder, not measured by us.
The use case
Support managers can only listen to a small share of calls. Each scorecard skill is broken into simple behaviours, such as 'the agent restates the problem'. Jev answers those questions about each agent turn in the transcript, and code combines the answers into a weighted score and coaching notes written by a person. Coaches rate a sample of the same calls so the team can see how far the scores agree with people. A coach reviews results before any feedback reaches an employee, and scores should not drive pay or discipline alone.
Example workflow
- Export call transcripts from the contact-center platform
- Define skills, rubric levels and true/false behaviours in a scorecard
- Ask Jev the behaviour and rubric questions for each agent turn
- Code combines answers into scores and attaches coaching tips
- Coach rates a sample of calls and compares agreement before sharing feedback
Tools
- Jev (TypeSafe AI)
- SQLite
- React
- Contact-center transcript export
Risks and guardrails
- Transcript errors lead to unfair scores
- Agreement with coaches must be checked before use
- Scores used for discipline need human review and consent rules
Sources
Written by Agents Lookup. Credits and official pages for the tools this use case names:
- GitHub repo transcript-scorecard by brandonbryant12 (seed calls are fictional): github.com/brandonbryant12/transcript-scorecard