Creating Legal Complaints through AI: comparing ChatGPT 3.5 and 4.0

Creating Legal Complaints through AI: comparing ChatGPT 3.5 and 4.0

Microsoft recently announced the launch of ChatGPT 4, and implemented it into its Bing search engine as a chatbot. The current freely available ChatGPT on the OpenAI website is ChatGPT 3.5, so downloading Bing and using the ChatGPT on it allows me to test the newer capabilities of ChatGPT 4, which includes access to the internet. I tried asking the exact same question I had asked earlier of ChatGPT, regarding filing a legal complaint against former President Trump on the recent issue of Stormy Daniels' payments, to the ChatGPT 4 that can be accessed through downloading Bing. 

From hereon, "ChatGPT' refers to ChatGPT 3.5 on OpenAI, while Bing's ChatGPT 4 refers to the newly launched ChatGPT 4. 

The first few questions seemed to give far less detailed responses: as you can see, the sample legal complaint is much shorter than what I had gotten from ChatGPT earlier. 

Compare this to ChatGPT's response: 

        As you can see, the ChatGPT response specifically lays out an Introduction, the Parties, the Factual Allegations, Claims for Relief, and so on. I questioned Bing's ChatGPT4 on this response, and after repeated questioning, it slowly and seemingly reluctantly gave more detailed response, yet they all fell short of the complete legal complaint sample I had gotten from ChatGPT. 

        At the end, I asked to give a complete sample instead of a short summary sample, and a fascinating thing occurred: ChatGPT 4 initially started to compose a fully detailed legal complaint, very similar and perhaps better than the one I had gotten from ChatGPT, yet halfway through writing it, it deleted its half-formed answer and instead refused to give an answer. When I tried repeating the same prompt, it shut down the thread. 

        I then refreshed the chat, and entered the exact same prompts once again. This time, ChatGPT4 on Bing fully generated the 'complete' sample legal complaint, although I did have to re-prompt it to finish the complaint as it initially stopped composing the sample half-way. 

        As this response shows, ChatGPT4 on Bing is not uniform in generating a response. It seems that it is still unsure when it is permissible to give a legal complaint form example, even if it is specifically clarified to be a mere sample which will not cause legal consequences. Bing's ChatGPT 4 notably generates allegations of breach of contract and defamation relating to the relationship between Defendant Donald J. Trump and Plaintiff Jane Doe (which would refer to Stormy Daniels in this case), while ChatGPT 3.5 created a more generic complaint where it lays out the facts, focusing on the violation of federal campaign finance laws as well as the emotional distress such conduct caused the Plaintiff and requested "any other relief this Court deems just and proper" without truly specifying the legal claim other than emotional distress. 

        This difference may be due to my prompt for Bing not including all the facts that I had copy-pasted from the linked website. Since the command prompt had a limit of 2000 words, I had linked the webpage itself and presumed Bing would access it or the internet and gain relevant facts from it to generate a response with. This may have resulted in the generated sample response being more focused on defamation and breach of contract. It may also be that ChatGPT 4 is more accurate in terms of providing a specific example of a legal complaint, rather than a generic guideline that ChatGPT 3.5 provided. 

        Further testing is required to see if repeated prompts continue to generate different legal sample complaints, and whether that initial problem of ChatGPT deleting its prompt while it was in the midst of writing it can be repeated. It should also be tested if feeding the facts of the case directly to Bing's ChatGPT similar to how I did with ChatGPT 3.5 will change the results. 

Bonus: 

        I also tried asking the chat thread on the side of Bing; despite using an identical prompt, this time the AI gave me a much less formal sample that read more like a letter to a judge. I am not sure if such a sample would really be what someone seeking to file a lawsuit would send. 

^ As can be seen above, this is more of an email making contact with the courts rather than an actual legal complaint. 

Thank you for reading, and I hope you found it informative and interesting.

If you are interested in discussing more about how ChatGPT can be used in such legal contexts, or have any questions, contact me at hello@simplawfy.ca 

Disclaimer: This story is only intended to be used for educational or recreational purposes. Responses by ChatGPT and similar AI chatbots, if mentioned in this story, should NOT be relied upon as factual. NO legal advice is being provided, and users must understand that there is NO attorney client relationship between you and the story publisher. The story should NOT be used as a substitute for competent legal advice from a licensed professional attorney in your state/country.