How to Use Large Language Models for Empirical Legal Research. Choi, J. H. August, 2023. ![How to Use Large Language Models for Empirical Legal Research [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper abstract bibtex Legal scholars have long annotated cases by hand to summarize and learn about developments in jurisprudence. Dramatic recent improvements in the performance of large language models (LLMs) now provide a potential alternative. This Article demonstrates how to use LLMs to analyze legal documents. It evaluates best practices and suggests both the uses and potential limitations of LLMs in empirical legal research. In a simple classification task involving Supreme Court opinions, it finds that GPT-4 performs approximately as well as human coders and significantly better than a variety of prior-generation NLP classifiers, with no improvement from supervised training, fine-tuning, or specialized prompting.
@misc{choiHowUseLarge2023,
address = {Rochester, NY},
type = {{SSRN} {Scholarly} {Paper}},
title = {How to {Use} {Large} {Language} {Models} for {Empirical} {Legal} {Research}},
url = {https://papers.ssrn.com/abstract=4536852},
abstract = {Legal scholars have long annotated cases by hand to summarize and learn about developments in jurisprudence. Dramatic recent improvements in the performance of large language models (LLMs) now provide a potential alternative. This Article demonstrates how to use LLMs to analyze legal documents. It evaluates best practices and suggests both the uses and potential limitations of LLMs in empirical legal research. In a simple classification task involving Supreme Court opinions, it finds that GPT-4 performs approximately as well as human coders and significantly better than a variety of prior-generation NLP classifiers, with no improvement from supervised training, fine-tuning, or specialized prompting.},
language = {en},
urldate = {2024-07-29},
author = {Choi, Jonathan H.},
month = aug,
year = {2023},
keywords = {machine learning, AI, empirical legal studies, natural language processing, artificial intelligence, computational analysis of law, large language models, LLMs},
}
Downloads: 0
{"_id":"A4BhuaD9MnQA67qXY","bibbaseid":"choi-howtouselargelanguagemodelsforempiricallegalresearch-2023","author_short":["Choi, J. H."],"bibdata":{"bibtype":"misc","type":"SSRN Scholarly Paper","address":"Rochester, NY","title":"How to Use Large Language Models for Empirical Legal Research","url":"https://papers.ssrn.com/abstract=4536852","abstract":"Legal scholars have long annotated cases by hand to summarize and learn about developments in jurisprudence. Dramatic recent improvements in the performance of large language models (LLMs) now provide a potential alternative. This Article demonstrates how to use LLMs to analyze legal documents. It evaluates best practices and suggests both the uses and potential limitations of LLMs in empirical legal research. In a simple classification task involving Supreme Court opinions, it finds that GPT-4 performs approximately as well as human coders and significantly better than a variety of prior-generation NLP classifiers, with no improvement from supervised training, fine-tuning, or specialized prompting.","language":"en","urldate":"2024-07-29","author":[{"propositions":[],"lastnames":["Choi"],"firstnames":["Jonathan","H."],"suffixes":[]}],"month":"August","year":"2023","keywords":"machine learning, AI, empirical legal studies, natural language processing, artificial intelligence, computational analysis of law, large language models, LLMs","bibtex":"@misc{choiHowUseLarge2023,\n\taddress = {Rochester, NY},\n\ttype = {{SSRN} {Scholarly} {Paper}},\n\ttitle = {How to {Use} {Large} {Language} {Models} for {Empirical} {Legal} {Research}},\n\turl = {https://papers.ssrn.com/abstract=4536852},\n\tabstract = {Legal scholars have long annotated cases by hand to summarize and learn about developments in jurisprudence. Dramatic recent improvements in the performance of large language models (LLMs) now provide a potential alternative. This Article demonstrates how to use LLMs to analyze legal documents. It evaluates best practices and suggests both the uses and potential limitations of LLMs in empirical legal research. In a simple classification task involving Supreme Court opinions, it finds that GPT-4 performs approximately as well as human coders and significantly better than a variety of prior-generation NLP classifiers, with no improvement from supervised training, fine-tuning, or specialized prompting.},\n\tlanguage = {en},\n\turldate = {2024-07-29},\n\tauthor = {Choi, Jonathan H.},\n\tmonth = aug,\n\tyear = {2023},\n\tkeywords = {machine learning, AI, empirical legal studies, natural language processing, artificial intelligence, computational analysis of law, large language models, LLMs},\n}\n\n","author_short":["Choi, J. H."],"key":"choiHowUseLarge2023","id":"choiHowUseLarge2023","bibbaseid":"choi-howtouselargelanguagemodelsforempiricallegalresearch-2023","role":"author","urls":{"Paper":"https://papers.ssrn.com/abstract=4536852"},"keyword":["machine learning","AI","empirical legal studies","natural language processing","artificial intelligence","computational analysis of law","large language models","LLMs"],"metadata":{"authorlinks":{}}},"bibtype":"misc","biburl":"https://bibbase.org/f/vr5ooa48xeYes5KDD/ailaw.bib","dataSources":["7FkfQdR6FwGXEAZFa","QHxajSYCsDY5s5PEr"],"keywords":["machine learning","ai","empirical legal studies","natural language processing","artificial intelligence","computational analysis of law","large language models","llms"],"search_terms":["use","large","language","models","empirical","legal","research","choi"],"title":"How to Use Large Language Models for Empirical Legal Research","year":2023}