AI Evals: Test AI Before You Trust It
Build an eval for an AI help assistant: a test set with known good answers, a scoring script, an AI grader checked against people, a comparison of two versions, and a release check that blocks a worse one.
Modules
6
Verified
0
In Progress
0
What you walk away with
- ✓A shareable portfolio URL with your project walkthrough
- ✓Module-by-module timeline of everything you built
- ✓All the code — pipelines, guardrails, deployment configs
- ✓Production patterns documented on your profile
A hands-on course for developers, product managers and QA. You work on Ask Corri, the help assistant of Corriwell Home Insurance (both made up): check 120 test questions for coverage, score answers with facts, rules and a graded check, compare a stand-in AI grader with 20 human grades, find that version B is better overall but worse on claims, run a short bias check by customer group, and set up a release check with a pass mark per topic. Plain Node scripts, deterministic by default. Educational, not legal advice.