Projects
-
Evaluation
Contract Tutors
Contract Tutors examines whether AI can answer student questions as reliably as law professors, especially when answers depend on reasoning rather than recall. In the blinded study, contracts professors compared AI-generated and faculty-written responses to questions students commonly bring to office hours. The professors preferred the AI responses in roughly three-quarters of the comparisons, including for hypotheticals and policy questions without a single correct answer. The project also introduces a way to evaluate AI tutors in fields where quality is defined by shared expert judgment.
-
Evaluation
JudgmentBench
In law and other judgment-rich domains, quality can be easier to recognize than to define. Is expert judgment more like a science, where quality can be broken into explicit criteria, or more like an art, where experienced practitioners can see what works even when they cannot fully explain why? JudgmentBench asks which of two methodologies–rubrics and comparative judgments–is better suited to assess quality in the legal domain. It brings together 30 real-world legal tasks and thousands of evaluations from experienced attorneys. The study finds that comparative judgments identify differences in quality more reliably and faster than rubric-based scoring. More broadly, it offers a framework for evaluating AI in fields where quality depends on professional judgment rather than factual correctness alone.
-
Legal Education
WitnessSim
WitnessSim develops an interpretable model of evolving witness behavior throughout a deposition. Rather than relying on a static AI persona, it represents the witness through several behavioral dimensions, including composure, knowledge, agreeableness, and rigidity, that respond dynamically to questioning. This allows both researchers and young practitioners greater control over how simulated witnesses behave under pressure. The project is a step towards leveraging AI-driven simulations for realistic and repeatable deposition training.
