JAKARTA - In recent weeks, a chatbot based on Artificial Intelligence (AI), ChatGPT has been tested on a number of tasks such as taking exams at law and business universities. This time, his latest challenge eliminates errors in a code.
The challenge givers are computer science researchers from Johannes Gutenberg University and University College London.
They managed to make ChatGPT can remove errors with sample code and fix them better than existing programs and are designed to do the same.
The researchers tested ChatGPT's performance using the QuixBugs bug fix benchmark. The automatic program repair system (APR) appears to be in an unfavorable position as it was developed before 2018.
Then, they provide QuixBugs 40201 to four different code improvement systems, namely ChatGPT, Codex, CoCoNUt, and Standard APR. Then manually, they check whether the suggested solution is correct or not.
On the first track, ChatGPT worked well with other systems. The bot made by OpenAI solved 19 problems, Codex solved 21, CoCoNUt solved 19, and the standard APR method found seven errors.
The success rate of ChatGPT with follow-up interactions found by the researchers reached 77.5 percent. However, Codex and ChatGPT have the most similar answers. This is not surprising, because ChatGPT and Codex are said by researchers to be from the same language model family.
However, the ability to chat with ChatGPT after receiving the initial answer made the difference, which ultimately led ChatGPT to solve 31 questions, and easily outperformed others, providing more static answers.
"The strong advantage of ChatGPT is that we can interact with systems in dialogue to determine demand in more detail," the researchers said in a new arXiv paper, which was first discovered by New Scientist.
"We are looking at most of our requests, ChatGPT is asking for more information on issues and bugs. By providing ChatGPT with such hints, its success rate can be further improved, correcting 31 of the 40 bugs," he added.
In addition, quoted from PCMag, Monday, January 30, ChatGPT can also quickly solve some problems, while others don't, "ChatGPT seems to have relatively high variations when fixing bugs. For end users, however, this means it can help to execute requests several times," said the researchers.
The English, Chinese, Japanese, Arabic, and French versions are automatically generated by the AI. So there may still be inaccuracies in translating, please always see Indonesian as our main language. (system supported by DigitalSiber.id)
