Ashurst pulls back the curtain on GenAI data from its latest trials

• In a market first, Ashurst's new report discloses the experimental data and in-depth insights of its exploration of generative AI in a bid to increase transparency and demystify its applications in the legal industry;
• Data indicates greatest initial value of GenAI is time saved in producing first drafts – participants reported an average of 45% of time saved, rising to 80% in some use cases;
• There is a direct correlation between the potential value of utilising GenAI for legal and its accuracy, however it can be difficult to distinguish between GenAI v human output with 50% of AI-generated material misidentified in a blind study;
• GenAI was viewed by 61% of respondents to be an effective support mechanism for managing workloads and creating opportunities to focus on higher value work;
• GenAI helped 88% of respondents to feel more prepared for the future.

Global law firm, Ashurst, offers a behind the scenes look at the firm's comprehensive trials of three different generative AI ('GenAI') platforms in its report, Vox PopulAI: Lessons from a global law firm's exploration of generative AI. The report is the first of its kind and discloses robust qualitative and quantitative data on Ashurst's evaluation of the use of GenAI in the legal sector. The study underscores the benefits of GenAI and provides guidance on how best to support people in the future adoption of this new technology capability.

The trials, which involved over 400 people spanning 23 offices and 14 countries, evaluated the use of a number of GenAI-powered tools, measuring their potential value and future impact in a legal context, as well as the best methods for navigating the exploration of GenAI.

The greatest initial value of GenAI for the legal sector was found to be helping lawyers to create drafts more quickly and efficiently. Through controlled experiments, the firm measured average time savings of 45% on creating first draft legal briefings increasing to 80% to draft corporate filings requiring review and extraction of information from articles of association. Translated into typical time spent, participants saved 2.5 hours per briefing draft – tasks that would normally take anywhere between a few hours to a few days to prepare. Producing first drafts in this way not only creates the opportunity for people to focus on higher-value work, but may potentially also lead to an increase in the speed that legal services are delivered.

While there is a direct correlation between the potential value of utilising GenAI for legal and its accuracy, Ashurst's research found human hallmarks in outputs appear to influence the perception of overall quality. In a blind study, an expert panel were asked to predict if a range of outputs were GenAI or human-generated. Interestingly, the panel correctly identified all outputs made by the firm's lawyers as human-generated. While half of the GenAI output was correctly identified as such, the other half was either mis-identified as human-produced or could not be determined by the panel. Given this dynamic, further research is required to fully understand and help organisations distinguish between the hallmarks of human versus GenAI produced work, and thus how to increase the perceived quality of Gen-AI powered outputs.

The value uncovered for GenAI during the trials was far broader than originally anticipated, as reflected by the 61% of post-trial survey respondents who felt that using GenAI would help them feel more supported in managing their workload. The research also showed that delivering and embedding a GenAI capability is about more than just saving time, it is also about preparing people to meet and stay ahead of market demands as they continue to evolve. Tellingly, the vast majority of respondents (88%) at the end of the biggest trial said that using GenAI technology helped them to feel more prepared for the future. 

Hilary Goodier, Global Head of Ashurst Advance, added:
"The buzz around AI has captured the imagination of so many in our firm, sparking conversations globally around how we can use AI to improve the way we work and deliver enhanced services to our clients. Our aim with these comprehensive structured trials was to go a step beyond simply identifying the best use cases and value GenAI can add, and to instead uncover the best approaches in supporting our people in the future adoption of this new technology capability.

"There is no ‘one size fits all’ when it comes to embedding GenAI appropriately. Our research has opened the door to a greater understanding of the digital literacy and development needs of our people and what is required to achieve broader digital transformation at Ashurst. We now have a better understanding of pain-points, user-needs and our lawyers' daily experience, which has allowed us to make investment decisions based on measurable data and to move forward with designing our own GenAI policies and implementing its use in our firm."