Every week brings a new headline: a threshold crossed, a benchmark bested. Despite the noise, every professional is asking themselves the same questions: am I using the right tool for the right job, honoring the assurances I've made to my clients, and following best practices that I can defend?
This webinar will help you build a practical checklist for evaluating AI at your firm, starting with the benchmarks themselves. We'll map the landscape you'll meet in vendor materials: broad knowledge tests, human-preference leaderboards, and the domain-specific evaluations now emerging in law, finance, and tax. Then we'll get into methodology: how these tests are built, scored, and graded, and why that determines whether a number means anything at all.
We'll also consider what benchmarks routinely overlook: citation accuracy as distinct from answer accuracy, the weighting of binding versus persuasive authority, currency as the law changes, completeness versus correctness, and whether a system admits uncertainty when it should.
You'll learn
- What common benchmarks actually measure, and where that diverges from what the headline number implies
- The questions that separate a robust evaluation from a persuasive chart
- How to assemble a golden set of your own questions — work whose answers you already know — and test every candidate against it
CPE Credit Information
Subject Area: Technology
Course Level: Overview
Instructional Method: Group Internet Based
Prerequisites: None
Advanced Preparation: None
