Posted: 16th November 2023

A joint research paper on the good, the bad and the unknown elements of ChatGPT has been awarded best paper by an internationally renowned digital forensics community. 

The DFRWS Conference in Singapore recently awarded best research paper to ChatGPT for digital forensic investigation: the good, the bad and the unknown.  

The paper, which was written by a Europe-based group of researchers, including University of Oxford Department of Computer Science lecturer Dr Christopher Hargreaves, assesses the impact and potential impact of ChatGPT on the field of digital forensics, specifically looking at its latest pre-trained Large Language Model, GPT-4.   

The paper includes eight experiments to explore the potential applications of ChatGPT for digital forensics and provides valuable insights.     

Many of the limitations identified are consistent with findings from other studies and existing system documentation. In particular, the phenomenon of ‘hallucination’, which nicely disguises the alternative term ‘incorrect’, is a recurring theme.     

Dr Hargreaves said: ‘The uses of ChatGPT for digital forensics are limited at the moment, since it would involve sending evidential data to a third-party, and it is essential that the output is correct. Our research has however illustrated the potential of LLMs in areas in which you can’t be wrong, such as generating scenarios for teaching, or where the output could be tested and shown to be effective, such as generating keyword lists, or assisting in code generation, provided the user has sufficient knowledge to evaluate, interpret, and correct the results. Not unlike the way that other automated tools in digital forensics should be used.'    

‘It is important to remember that despite the hype and sometimes impressive capabilities, this technology is still rather new. There is cause for concern if it is overused, but it does show great potential for the future. Like all automation for digital forensics, it is useful, but requires caution and competent human oversight.' Department of Computer Science lecturer Dr Christopher Hargreaves  

The other researchers included:  

• Mark Scanlon, Forensics and Security Research Group, School of Computer Science, University College Dublin, Ireland  
• Frank Breitinger, School of Criminal Justice, University of Lausanne, Lausanne, Switzerland  
• Jan-Niclas Hilgert, Fraunhofer FKIE, Bonn, Germany  

• John Sheppard, Department of Computing and Mathematics, South East Technological University, Waterford  

The full paper is available: ChatGPT for digital forensic investigation: The good, the bad, and the unknown - ScienceDirect  

NB: While ChatGPT was the focus of the research conducted as part of this paper, it did not contribute to the paper’s content or analysis other than where directly quoted or described.