Accurate, Focused Research on Law, Technology and Knowledge Discovery Since 2002

Category Archives: AI

Evaluating Generative AI for Legal Research: A Benchmarking Project

Via LLRX – Evaluating Generative AI for Legal Research: A Benchmarking Project – It is difficult to test Large-Language Models (LLMs) without back-end access to run evaluations. So to test the abilities of these products, librarians can use prompt engineering to figure out how to get desired results (controlling statutes, key cases, drafts of a memo, etc.). Some models are more successful than others at achieving specific results. However, as these models update and change, evaluations of their efficacy can change as well. Law Librarians and tech experts par excellence, Rebecca Fordon, Sean Harrington and Christine Park plan to propose a typology of legal research tasks based on existing computer and information science scholarship and draft corresponding questions using the typology, with rubrics others can use to score the tools they use.

Nvidia denies pirate e-book sites are “shadow libraries” to shut down lawsuit

Ars Technica: “Some of the most infamous so-called shadow libraries have increasingly faced legal pressure to either stop pirating books or risk being shut down or driven to the dark web. Among the biggest targets are Z-Library, which the US Department of Justice has charged with criminal copyright infringement, and Library Genesis (Libgen), which was… Continue Reading

Google Researchers Say AI Now Leading Disinformation Vector

404 Media – and Are Severely Undercounting the Problem – “As an endless stream of entirely wrong and sometimes dangerous AI-generated answers from Google are going viral on social media, new research from Google researchers and several fact checking organizations have found that most image-based disinformation is now AI-generated, but the way researchers collected their… Continue Reading

RAG and Hallucination-Free Legal Research Tools

Github Blog: “What is retrieval-augmented generation, and what does it do for generative AI? Here’s how retrieval-augmented generation, or RAG, uses a variety of data sources to keep AI models fresh with up-to-date information and organizational knowledge.” Via LinkedIn – An Honest conversation about RAG and Hallucination-Free Legal Research Tools – “…We are smack-dab in… Continue Reading

Google scrambles to manually remove weird AI answers in search

The Verge: “…Google continues to say that its AI Overview product largely outputs “high quality information” to users. “Many of the examples we’ve seen have been uncommon queries, and we’ve also seen examples that were doctored or that we couldn’t reproduce,” Google spokesperson Meghann Farnsworth said in an email to The Verge. Farnsworth also confirmed… Continue Reading

How to Detect and Guard Against Deceptive AI-Generated Election Information

Brennan Center – “Time-tested fact-checking practices will help limit the effectiveness and spread of misleading election information. Generative artificial intelligence is already being deployed to mislead and deceive voters in the 2024 election, making it imperative that voters take steps to identify inauthentic images, audio, video, and other content designed to deceptively influence their political… Continue Reading

Best Practices for Using AI When Writing Scientific Manuscripts

ACS Nano 2023, 17, 5, 4091–4093 Best Practices for Using AI When Writing Scientific Manuscripts Caution, Care, and Consideration: Creative Science Depends on It. February 27, 2023 https://doi.org/10.1021/acsnano.3c01544 – “Science is communicated through language. The media of language in science is multimodal, ranging from lecturing in classrooms, to informal daily discussions among scientists, to prepared… Continue Reading

Here are the companies OpenAI has made deals with to train ChatGPT

Fast company: “OpenAI’s chatbots scored a big new data source following the company’s deal with News Corp. on Wednesday. With the stroke of a pen, ChatGPT and the company’s other services added the Wall Street Journal, New York Post, MarketWatch, Barron’s, and other publications to its database. The deal, which did not include Fox News… Continue Reading

AI on Trial: Legal Models Hallucinate in 1 out of 6 Queries

Stanford University Human-Centered Artificial Intelligence – A new study reveals the need for benchmarking and public evaluations of AI tools in law. “Artificial intelligence (AI) tools are rapidly transforming the practice of law. Nearly three quarters of lawyers plan on using generative AI for their work, from sifting through mountains of case law to drafting contracts… Continue Reading

Here’s what’s really going on inside an LLM’s neural network

Ars Technica:  “With most computer programs—even complex ones—you can meticulously trace through the code and memory usage to figure out why that program generates any specific behavior or output. That’s generally not true in the field of generative AI, where the non-interpretable neural networks underlying these models make it hard for even experts to figure… Continue Reading