The adoption of AI tools in the programming world has skyrocketed, enabling coders to generate large amounts of code with simple text prompts.S,
earlier this year, Google found it 90 percent of software developers across the industry are using AI tools at work, up from just 14 percent last year.
But all that convenience comes with some glaring drawbacks. The tools have repeatedly been found to be unreliable and inaccurate, leading to mistakes and even requiring some programmers to work for long periods of time to identify and correct them.
Adding to the reality check, a new report AI software company CodeRabbit found that AI-generated code was far more error-prone than human-written content – and by a significant margin. Of the 470 pull requests analyzed by the company, the AI code generated an average of 10.83 points per request, while the human-written code generated only 6.45 points.
In other words, the AI code caused 1.7 times more problems than the human code, which once again highlights the major weaknesses of generative AI tools.
“Result?” CodeRabbit has come to this conclusion in its report. “Clear, measurable, and consistent with what many developers are intuitively feeling: AI speeds up output, but it also increases certain categories of mistakes.”
Worse, the company found that the AI-generated code generated higher rates of “critical” and “major” issues, leading to a “meaningful increase in critical concerns that attracted reviewers’ attention.”
The AI code was also most likely to contain errors related to logic and correctness. However, CodeRabbit found the biggest weaknesses in code quality and readability, which are issues that “can slow down teams and turn into long-term technical debt.”
Then there are the serious cybersecurity concerns, with generated code posing issues related to improper password handling that could expose protected information among other insecure practices.
On the positive side, CodeRabbit found that the AI code was adept at keeping spelling errors to a minimum. Humans were twice as likely to misspell.
This is not the first time we have heard about flaws in AI-generated code. one in september reportManagement consultants Bain & Company concluded that despite being “one of the first areas to deploy generative AI”, programming had “savings not been significant” and “results that did not live up to the hype.”
Security firm Apiro also found in its research that developers who used AI created ten times more security problems than their counterparts who did not use the technology.
As a result, the programmer is forced to select the generated code to ensure that no serious problems occur. according to a july study From the non-profit Model Evaluation and Threat Research, programmers were actively being slowed down by AI assistance tools when they worked without them.
In short, while companies made sky-high promises about technology making programmers’ lives much easier, the reality looks far more nuanced. CodeRabbit’s report suggests that there may soon be a change in the types of tasks human developers are required to perform – such as solving issues being introduced by error-prone AI coding tools.
“These findings reinforce what many engineering teams have felt about 2025,” said David Locker, director of CodeRabbit AI. statement“AI coding tools dramatically increase output, but they also introduce predictable, measurable vulnerabilities that organizations must proactively mitigate,”
More on AI Programming: Report finds AI coding is being over-hyped
