Moderate product reviews before they appear on your store
New product reviews are checked by a decision model against your posting rules, so clearly fine reviews publish on schedule and anything possibly abusive, off-topic, or containing personal details waits for a person.
Author's figure: Author's figure: about $0.00003 per post judged on eight questions (firehose-judge). Reported by the builder, not measured by us.
The use case
Stores that publish customer reviews need to keep out spam, abuse, and personal details, but should not hide honest negative reviews. Jev reads each review and answers a set of yes-or-no questions: is it about the product or service, does it contain abusive language, does it include personal contact details, and does it look like spam. Reviews where every answer clears a 0.9 confidence and no rule is broken are approved for the normal publishing queue. Anything that trips a rule or is unsure is held for a person. Jev never removes a review because it is negative. A team member reviews every held item and decides whether to publish, edit out personal details, or reject it.
Example workflow
- A new review arrives in your review app, such as Judge.me, and a webhook sends it to Make.
- The review text, star rating, and product name are sent to Jev as the state.
- Jev answers Noul questions: the review is about the product or service, it contains abusive or hateful language, it includes a phone number, email, or address, and it looks like spam or an advert.
- Code approves the review for publishing only when it is on-topic and clear of every rule at the threshold.
- Held reviews are posted to Slack with the rule that tripped, and a person decides what to do.
- A monthly check compares held and published reviews by star rating to make sure negative reviews are not being held more often.
Tools
- Jev (TypeSafe AI)
- Shopify
- Judge.me
- Make
- Slack
- Google Sheets
Risks and guardrails
- Holding honest negative reviews would mislead buyers — guardrail: star rating is never a reason to hold, and hold rates by rating are checked monthly.
- Sarcasm can be read as abuse — guardrail: abusive answers below a high probability are held for a person rather than rejected.
- Review platforms and local rules may restrict moderation — guardrail: check your platform's terms and publish your moderation policy.
Related recipe
Support Ticket and Review Triage with Policy-Cited Draft Replies: An agent tags every helpdesk ticket and public review by intent, urgency, and sentiment, routes it, and drafts a reply that cites your help center. People approve refunds and all public replies.
Want it set up for you? Request a quote.
Sources
Written by Agents Lookup. Credits and official pages for the tools this use case names:
- jevmod by Omar Hernandez on Made with Jev: madewithjev.com/builds/jevmod
- jevmod: original post or repo by Omar Hernandez: github.com/ohernandezdev/jevmod
- firehose-judge by Leo Mata on Made with Jev: madewithjev.com/builds/firehose-judge
- firehose-judge: original post or repo by Leo Mata: github.com/ragelink/firehose-judge
- Shopify Dev MCP server (AI Toolkit) (official page): shopify.dev/docs/apps/build/ai-toolkit (checked 2026-11-09)
- Make MCP server (official page): developers.make.com/mcp-server (checked 2026-11-09)
- Slack MCP server (official page): docs.slack.dev/ai/slack-mcp-server (checked 2026-11-09)