Accurate, Focused Research on Law, Technology and Knowledge Discovery Since 2002

Stanford HAI Tests Westlaw But The GenAI Results Look Worse

Artificial Lawyer: “Ok this story is getting into unusual territory now. Artificial Lawyer just got an email from the spokespeople for the Stanford University HAI team who told this site the researchers had updated their genAI study of hallucinations in case law tools to include Thomson Reuters’ Westlaw. And guess what….? Westlaw has come out even worse than the Practical Law tests (see below) according to what they have published in an updated paper. Here is the new statement to AL from HAI: ‘Letting you know that the research and blog post have been updated with new findings. The study now includes an analysis of Westlaw’s AI-Assisted Research alongside Lexis+ AI and Ask Practical Law AI.’ They have updated the HAI group’s findings here to reflect this. As you may remember, this whole thing started when a group of researchers tested whether LexisNexis’s and Thomson Reuter’s genAI tools were as good as hoped for case law research. There was plenty of confusion caused when the team tested Practical Law, rather than Westlaw for the case law questions. They have since been given access to Westlaw and hence the new results…Here is the link to the original story in Artificial Lawyer, and there are two more articles with comments that follow it that give more context – please see the AL site…”

See also Thomson Reuters: GenAI Tool Tested By Stanford DID ‘Leverage Casetext’

Sorry, comments are closed for this post.