I'm Scared! ChatGPT Successfully Graduates Examination At The University Of Law And Business

Professor Jonathan Choi at the University of Minnesota Law School, testing chatGPT. (photo: twitter @JonathanHChoi)

JAKARTA - OpenAI's chatGPT recently passed the legal exam after writing essays on various topics ranging from constitutional law to taxation and loss.

The chatbot was able to complete four courses at the University of Minnesota and other exams at theWArton School of Business University of Pennsylvania.

To test how well ChatGPT can produce answers on exams for four courses, professor Jonathan Choi at the University of Minnesota Law School recently filed a pile of questions on the bot.

Bot managed to solve 95 multiple choice questions and 12 essay questions, the bot got a C+ score overall by reaching a low score in its class.

"In writing essays, ChatGPT shows a strong understanding of basic law rules and has a solid organization and composition consistently," said the professors in a white book entitled ChatGPT go to law school, published earlier this week.

🚨🚨 new white paper! We used ChatGPT to write exams and graded them blind alongside real law students at Minnesota. ChatGPT got a C+ average and a low but passing grade in Con Law, Employee Benefits, Tax, Torts. 1/5 https://t.co/sN8uW4qtnR

— Jon Choi (@JonathanHChoi) January 24, 2023

"(However, bots) often have difficulty finding problems when given an open prompt; core skills in legal school exams," he added.

The purpose of this test is, said Choi, to explore the potential of ChatGPT to help lawyers in their practice and assist students in the exam, either permitted or not by their professors, because their questions often mimic the writings of lawyers in real life.

ChatGPT struggles with the most classic legal school exam components, such as finding potential legal problems and in-depth analysis that apply legal rules to the facts of a case. But ChatGPT can be very helpful in producing the first draft that students can then enhance," Choi told CNN International quoted Friday, January 27.

He argues that AI's human collaboration is the most promising use case for ChatGPT and similar technology. Choi suggested that the technology could be a valuable teaching aid.

Overall, ChatGPT is not a great law student who acts alone. But we hope to collaborate with humans, language models like ChatGPT will be very useful for law students who take exams and for lawyer practices," said Choi.

Meanwhile, ChatGPT fared better during a business management course exam at theWArton School of Business University of Pennsylvania, where he earned a B to B-.

In a paper detailing its performance, Christian Terwiesch, a Whalen business professor, said ChatGPT did an outstanding job in answering basic operating management questions and process analysis, but had difficulties with more sophisticated clues and made surprising mistakes with basic mathematics.

However, chatbots are very good at modifying the answer in response to human instructions, such as reworking answers after showing errors, suggesting potential people to cooperate with AI.

Unfortunately, recently officials in New York and other jurisdictions have banned the use of ChatGPT in schools. Given that ChatGPT appears above average on its exams, Terwiesch agrees that restrictions should apply to students when they take the exam.

"Arangan dibutuhkan. Apalagi, ketika Anda memberikan gelatan dokter, Anda ingin mereka tahu obat, bukan cara menggunakan bot. Hal yang sama berlaku untuk sertifikasi keskill lainnya, termasuk hukum dan bisnis," tutur Terwiesch.

For information, ChatGPT was launched to the public at the end of November 2022, but has created a sensation with credible results in self-made natural language writing, although OpenAI has explained the software is still in process.