State of AI in eDiscovery Report
State of AI in eDiscovery
Five Focus Areas Shaping the AI Legal Landscape
1st Edition Benchmark Survey, 2024
Foreword
2023: Generative AI’s 1st Year
The widespread introduction of generative AI (Gen AI) in 2022 caused unprecedented disruption across the world. The unique user accessibility offered by Gen AI chat tools gave many people their first experience with the power and accuracy of Large Language Model (LLMs)—ushering in an AI renaissance across virtually every industry.
This renaissance has been continually spurred on by subsequent releases of Gen AI technology (and news of potential regulatory restrictions) throughout 2023.
In November of 2022, OpenAI released ChatGPT. The technology reached an unprecedented one million users within the first 5 days of its release. By January of 2023, this number had surged to 100 million users.
The next month saw an avalanche of new Gen AI releases by competing companies in the tech industry. In February alone, Microsoft announced it had integrated the technology into its search engine (Bing), Google announced the impending release of its own Gen AI chatbot (Bard), and Meta unveiled its Gen AI LLM model (LLaMA). In March, OpenAI released an upgraded version of ChatGPT (ChatGPT-4), which included never-before-seen reasoning capabilities, while Microsoft announced it would integrate Gen AI capabilities into the wildly popular M365 platform with its Copilot application.
Regulatory Response
At the same time, regulators across the world were also keying in on AI, spurred on by concerns around possible security, privacy, and ethical risks presented by the advancement of Gen AI technology.
In March of 2023, the U.S. Copyright Office launched an initiative to examine content generated by AI and Italy temporary banned ChatGPT. In April, the EU proposed the Artificial Intelligence Act, seeking a way to regulate uses of the technology within its purview.
Throughout the year there were also a host of lawsuits related to Gen AI technology development and use—alleging everything from copyright infringement to defamation claims against technology developers for AI-generated content.
Federal and state court judges also began cracking down on the technology—issuing orders and mandates that restricted and/or dissuaded attorneys from using AI and Gen AI in court room filings.
Legal Industry Impacts
Against that backdrop, attorneys largely took a “wait and see” approach to Gen AI adoption for legal work (with some notable exceptions) throughout 2023.
However, the technology still had a tidal wave impact on the legal field as a whole—and eDiscovery specifically. In-house corporate legal teams across industries grappled with a surge of business units (and individual employees) seeking to test and use AI capabilities to reduce manual labor and increase efficiency.
Corporate teams looked to their outside counsel to help them understand and control the impacts and risks the new technology posed. Law firms felt this pressure and sought out technical expertise to help them counsel clients on the nascent technology, while working to stay abreast of potential new AI regulations that would impact clients. At the same time, attorneys working within law firms looked for secure ways to take advantage of Gen AI to make their own practice more efficient.
1st Edition AI in eDiscovery Report
As an industry leader in AI, it is imperative that Lighthouse continue to keep a close pulse on the fast-paced evolution of AI and Gen AI within the eDiscovery and legal industries—both for our own strategic AI development roadmaps and as a service to our clients. With this in mind, Lighthouse is releasing the 1st edition report of our new AI in eDiscovery benchmarking survey initiative.
The report was compiled from a highly curated survey of 250+ eDiscovery and legal experts, with a roughly even split between law firms and corporate respondents. This new annual survey is designed to provide a strategic and holistic view of where eDiscovery and legal experts currently stand with AI, as well as where they’re headed in the coming year.
Executive Summary
Summary of Findings
- Survey respondents consisted of 268 eDiscovery experts, with a roughly even split between law firm (56%) and corporate (44%) respondents.The majority of law firm respondents came from firms of +500 attorneys. Similarly, corporate respondents came from companies of +$500M in annual revenue. Over half of the experts had more than 6 years of experience in their specific field.
- Across this group there is broad market interest in AI—with 87% of those surveyed citing they are “interested” or ”very interested” in AI. However, many are still undecided about the positive or negative impacts for legal with nearly 2 in 5 respondents citing a neutral / undecided attitude about AI impacts specific to legal.
- The surveyed audience was generally familiar with AI used in the practice of law with about 75% of responses citing they are "slightly familiar" to "familiar," with relatively few reporting deep familiarity (11-12%) or no familiarity at all (15-16%).
- Currently, AI adoption at the company level is still relatively low with just 1 in 5 respondent companies actively using AI, today; however, 2 in 5 are currently evaluating options.
- Regardless of direct experience with AI, views on both opportunities and risks were similar across all respondents with key themes emerging around the opportunity to enhance productivity and scalability and concerns relating to data privacy, security, and ethical considerations.
- Overall, respondents are more comfortable utilizing AI to perform automated, repetitive, or data intensive analysis—but less comfortable utilizing AI for more human nuanced work such as legal writing or developing case strategy.
- The largest barrier to increasing perceived efficacy of AI solutions is the need for constant verification. Improving users confidence in the accuracy of AI outputs through the continued refinement and improvement of algorithms will be critical to realizing the value of AI for legal and eDiscovery uses.
Key Themes
Six Themes Related to AI and Legal Emerge Across Respondents Regardless of Demographic Factors, Relative Experience with AI, and Perception of Opportunity or Risk
Data Privacy and Security
Respondents frequently mention concerns about data privacy and the security of client information as AI integrates into legal practices. This indicates a priority for ensuring AI solutions are compliant with data protection regulations and can maintain attorney-client privilege.
Efficiency and Cost Reduction
Many responses highlight the potential for AI to increase efficiency in legal research, document review, and contract management. This suggests a market demand for AI tools that can streamline routine tasks and reduce labor costs.
Ethical and Legal Considerations
Ethical dilemmas, such as the potential displacement of legal professionals and the reliability of AI-generated work, are significant concerns. There is a need for clear guidelines and standards to govern the ethical use of AI in the legal industry.
Accuracy and Reliability
The accuracy of AI in legal tasks, such as drafting documents and conducting research, is a recurring theme. Market opportunities may exist for AI solutions that can demonstrate high levels of precision and dependability.
Acceptance and Regulation
Responses indicate a cautious approach to AI adoption due to uncertainties about court acceptance of AI-generated work and the lack of industry regulation. This suggests a potential for services that help legal firms navigate the regulatory landscape and integrate AI responsibly.
Impact on Jobs
There is apprehension about AI's impact on employment within the legal sector, particularly regarding routine and repetitive tasks. This could lead to a shift in the job market and a need for retraining and redeployment of legal staff.
Five Focus Areas for AI in the Legal Industry
Overall, the legal industry is actively exploring AI—if they haven’t already implemented it. But they are cautious of security/privacy impacts, as well as concerned with the accuracy of AI-generated results. This coincides with the higher risk inaccurate results can pose to legal practitioners (i.e., the potential for errors to cause irreparable harm to a law firm’s reputation, incur sanctions on offending parties, damage the outcome of a litigation, etc.). Looking ahead, five key strategic priorities emerge to support expanded adoption of AI in the legal industry.
- Develop AI solutions with robust data security features to address privacy concerns.
- Create AI tools that enhance efficiency in legal processes while ensuring high accuracy.
- Advocate for the establishment of ethical standards and regulations for AI in the legal industry.
- Educate the legal community about the benefits and limitations of AI to foster acceptance.
- Monitor the job market and provide support for skill development in response to AI adoption.
Demographics
Respondent Demographics
Law Firm Roles
Corporate Roles
Familiarity and Usage
While 1 in 5 companies are currently using AI, and 2 in 5 companies are actively assessing use, the perception amongst those actively using still indicates some skepticism about the efficacy of their AI solutions.
Effectiveness: How would you rate the effectiveness of the AI solutions currently used in your organization
Sample size = 53; total sample size = 268
Legal AI solution adoption is being driven by individual use with 50% of respondents claiming personal use of AI for work, but just 20% reporting company adopted AI technology.
Which of the following solutions is your company using?
Sample size = 74 ; total sample size = 268
1 in 5 respondents work at companies that are currently using AI-based solutions and out of those, almost two-fifths use either Microsoft Co-Pilot or ChatGPT Enterprise. Almost 3/10 respondents said that they use a different tool than those listed.
Which of the following solutions are you personally using for work?
Sample size = 268
48% of respondents said they are personally using legal AI tools, but there is wide variance in the tools being used.
Co-Counsel and aiR by Relativity are the most prevalent accounting for 25% of legal AI tool usage by this group, but there’s a long tail of other tools in use.
Regardless of higher usage of legal AI tools by the respondent group, the stated levels of familiarity with the impact of AI tools in the practice of law or on information governance are relatively similar with about three quarters (74%–72%) being slightly familiar to familiar.
How familiar are you with AI tools used in the practice of law? (eg. Harvey AI, Lexis+ AI,Co-Counsel, Relativity aiR, etc.)
Sample size = 268
How familiar are you with the impacts of business AI tools on areas such as compliance, eDiscovery, and information governance?
Sample size = 268
Regardless of level and type of exposure to AI (for legal or more generally for enterprise productivity) respondents view the potential to positively impact legal work as significantly higher priority than understanding the impacts of AI use on data governance issues.
Which of the following areas do you believe is the highest priority when it comes to AI usage related to legal?
Sample size = 268
Attitudes and Motivations
Respondents are generally interested in AI; however, the lack of awareness by many about specific company policies and positions toward AI, highlights the lack of a common company point of view overall.
Top motivations for those interested in, or actively using, AI relate to: positive productivity impact, the ability to focus attorney time more strategically, and improved quality of work product.
Which of the following are your top motivations when using AI technologies or solutions?
Sample size = from 0 to 204; total sample size = 268
Following closely behind impact on work, were motivations around operational efficiencies—highlighting the pressure on both corporations and law firms to more effectively manage spend and ROI.
Top areas of perceived risk related to the use of AI center on accuracy and data privacy and security concerns.
Top drivers of risk mitigation align directly to top areas of perceived risk:
- Data security/privacy measures from the providers (50%)
- Improvement in accuracy of the AI (47%)
- Data security/privacy measures by the company of employment (40%)
Size of law firm directly related to differences in perceived risk areas by respondents:
- Those from large firms saw more risk associated with data security than those from mid-size firms (70% vs. 61%)
- Those from large firms saw more risk associated with reliability than those from mid-size firms (75% vs. 51%)
- Those from large firms saw less risk associated with job displacement than those from mid-size firms (32% vs. 47%)
Qualitative responses reflect the need to balance both the opportunity presented by AI with the many legal risks potentially introduced.
“AI will enhance legal research. AI will decrease the amount of time it takes to analyze extensive legal databases and extract information expediently. AI raises ethical concerns including data protection and privacy.”
“The law is a human institution rather than an exact science, and having any meaningful legal analysis done by AI would likely contain errors/loopholes.”
“...data privacy and the reliance upon a technology that hasn’t fully demonstrated its trustworthiness...”
“AI adoption can strongly enhance discovery efficiency, particularly in the organization and review of large-scale document production. Strong oversight of the implementation process must be integrated into policy, to avoid potentially catastrophic breach of security, confidentiality, client privilege, etc.”
Opportunity and Impact
Level of trust in AI vary depending on use case with the highest areas of trust relating to repetitive and data-intensive tasks, while there is a lower amount of trust for areas of more nuanced execution.
There is particular opportunity for AI use in eDiscovery, with roughly half of respondents citing comfort or interest in at least 7 different use cases.
Which of the following uses of AI are you most interested/comfortable using to support your eDiscovery activities? (Please select all that apply.)
Sample size = 239; total sample size = 268
Respondents identify the need to verify results as a significant barrier to full impact and highlight the need for further refinement and development of the algorithms to improve accuracy.
“...the algorithms need some ironing out...”
”...the output is good but not yet excellent...”
“...occasional errors occur...”
“...accuracy needs to be improved for saving time on cross-checking...”
“We are in a heavily regulated industry so overall tends to be more conservative in adopting the newest AI technology.”
“The tools are effective but the company needs to continue to evaluate the deployment of the tools. The tools need to be deployed on a broader basis to be more effective.”
“Everything AI accomplishes must be reviewed and vetted for accuracy and completeness.”
Conclusion
Though it’s been barely a year since ChatGPT took the world by storm, popularization of generative AI will have widespread impact across all industries. The first annual Lighthouse State of AI in eDiscovery survey results enforce our initial outlook that the impact for those in the legal profession has been twofold: that Gen AI opens up brand new ways to realize value and accelerate outcomes, while posing novel challenges that prompt key questions around security, compliance and managing AI data types.
The interest indicated by respondents points to the rapidity with which we might expect to feel the impacts of widespread AI adoption. Early indications imply that the pace of AI adoption may fly in the face of what has historically been an industry quite slow to adopt new technologies—with 50% of individual contributors pioneering AI for work, personally. It stands that growth in formalized company strategies must quickly follow.
2023 was a year of interest, experimentation, and great expectations. Lighthouse will continue to keep a close tab on how experience and perceptions change in 2024—as increased availability and access to AI at work give people real-world experience with the promise and potential pain associated with this revolutionary technological moment across industries.
We will continue to conduct this survey annually and are excited to see how the impact of AI continues to drive innovation and evolution for legal and eDiscovery. In the meantime, we’d love to hear what you and others are doing and thinking about in terms of AI innovation, use, and impact in the legal field. If you have questions about this report or you’d like to speak with one of our experts about AI, send us an email at insights@lighthouseglobal.com.
2024 State of AI in eDiscovery Report
Findings in this report were based on a survey of 268 eDiscovery experts from across large corporations (+$500M in annual revenue) and large law firms (+500 attorneys).
Fill out the form to access the full report and learn:
- How those in the eDiscovery industry are currently leveraging AI, including specific tools and specific use cases
- Hurdles to AI adoption by eDiscovery professionals and their perspectives on how these barriers can be overcome
- Immediate and long-term concerns related to the risks and impacts of AI for legal, including the best ways to mitigate those risks