AI Agents, Evaluation, and Model Change
Compare AI workflows by observed errors, evidence and limits—not by impressive demos or model names.
For students
Your outcome: A model-evaluation card, hypothetical role limits, a new corpus case and an explanation of false-pass risk.
Files, templates and reference guides
Use the lab to determine which files to give your AI assistant. This list is a navigation aid, not permission to load every resource into a model.
- fixtures/model-cases.jsonfixtures/model-cases.json
- Model / Tool Evaluation Cardtemplates/model-evaluation-card.md
- Product Studio — build something you want to existCOURSE_PROJECT.md
- Lab help — one step at a timeLAB_HELP.md
- Run the academyLAB_SETUP.md
- Mac, Linux and Windows lab guidePLATFORM_GUIDE.md
- Your product, your repository, a professional workflowPROJECT_GITHUB.md
- Small models, serious work, modest budgetSMALL_MODEL_GUIDE.md
- Submit, review and improve your workSUBMISSIONS.md
- fixtures/mock-model-outputs.jsonfixtures/mock-model-outputs.json
- Model evaluation protocolfixtures/model-evaluation-protocol.md
- Product milestone — Session NNtemplates/project-milestone.md
- Product review — useful feedback, no pointstemplates/project-review.md
- Session 11 — Model Evaluation, one step at a timewalkthroughs/11-model-olympics.md
Homework and submission
Follow the homework in this session’s lab. Your course-long product is separate from practice exercises. Use the submission guide and your learning path to track the next step.
For instructors
Public teaching resources—not a private staff area. Keep learner records and individual feedback elsewhere. Answer keys and reviewer labels must stay out of the context of a model being evaluated.
- Lesson 11 — AI Agents, Evaluation, and Model Changelessons/11-ai-agents-and-model-change.md
- Instructor GuideINSTRUCTOR_GUIDE.md
- Instructor answer key — use during debriefinstructor/ANSWER_KEY.md
- instructor/model-labels.jsoninstructor/model-labels.json
Pre-open the lab, slides, required evidence and templates. Rehearse the demo and keep a clearly labeled fallback ready. Protect learner practice time; move unrelated setup troubleshooting into coaching.
Recordings and transcripts
Class recordings and transcripts are restricted to enrolled learners and authorized guests. They are not published on this public page. No protected replay is linked here yet; ask the instructor for authorized access. Private presentation reviews and individual feedback are not public teaching materials.