At least one job appears to be safe from artificial intelligence, as a new study found that ChatGPT is actually not that good at advanced mathematics, assuaging accountants about their future employment prospects.
The meteoric rise of generative AI platforms like ChatGPT have raised a number of concerns about potential job displacement. In fact, some studies have found that up to 80% of jobs will be impacted by artificial intelligence, which has the potential to wreak absolute havoc on the economy.
Luckily for accountants, though, a new study has found that ChatGPT — the best and most popular generative AI platform available today — struggles when it comes math.
ChatGPT Can’t Pass Math Tests
In a study from Bringham Young University (BYU), researchers found that ChatGPT is still worse than actual accounting students at answering test questions. The generative AI platform scored a lowly 47.4% on exams common in accounting classes, while students averaged a score of 76.7% across the board.
The study was quite expansive, with more than 25,000 students participating across 186 colleges and universities in 14 different countries. The exam in question covered everything an accountant would need to know, including accounting information systems (AIS), auditing, financial accounting, managerial accounting and tax.
On top of all that, the exam was purposely varied on type and difficulty of the questions, which proved even more difficult for ChatGPT. The generative AI platform did fine on true/false questions (68.7%) and multiple-choice questions (59.5%), but struggled mightily on short-answer questions, netting scores between 28.7% and 31.9% in that category.
Can ChatGPT Pass Other Tests?
So ChatGPT can’t pass advanced mathematics tests, who cares? The generative AI can perform a lot of other tasks quite well, and it probably can’t pass any other advanced education tests, right? Wrong!
According to OpenAI, the company behind the innovative technology, ChatGPT is actually quite impressive when it comes to passing these kinds of exams. The generative AI passed the bar in the 90th percentile, received a nearly perfect score on the GRE Verbal test, and passed 13 of 15 AP tests currently available to students.
So, are teachers worried about cheating with this kind of technology? Of course they are, but that discussion has been common for years.
“When this technology first came out, everyone was worried that students could now use it to cheat. But opportunities to cheat have always existed. So for us, we’re trying to focus on what we can do with this technology now that we couldn’t do before to improve the teaching process for faculty and the learning process for students. Testing it out was eye-opening.” – David Wood, lead study author and a BYU professor of accounting.
The Future of AI
The break-neck speed at which generative AI platforms like ChatGPT have rolled out and improved productivity have raised concerns among those that believe society is not fully prepared for the impacts of robust AI use across the world.
Still, while this study may prove that one job is safe for the time being, its authors understand that this means we need to take a long hard look at how we do things across the educational and business landscape.
“It’s an opportunity to reflect on whether we are teaching value-added information or not. This is a disruption, and we need to assess where we go from here.” – Melissa Larson, study coauthor and fellow BYU accounting professor
Simply put, AI is going to change the world in one way or another, and if you think your job isn’t going to be impacted in the long run, you’ve got another think coming.
