Twitter's Changes In API Fails, Lack Of Engineers And Old Technology Is The Cause
Twitter's website is experiencing new and hard to fix problems. Although the company managed to recover from the latest disruption in a few hours, the story behind how it happened suggests there will likely be similar problems in the future.
On Monday morning, March 6, Twitter users discovered a number of connected issues. Clicking on the link no longer opens them; vice versa, users will see a mysterious error message that reports that "the current API plan does not include access to this endpoint." Images are also no longer open. Other users report that they cannot access TweetDeck, Twitter's clients for professional users.
Chaos controls the timeline, when users tweet hard about the annoyance - often depicting their points with images that no one can see because they will not open.
In a tweet, the company offered a very general explanation of what happened.
Some parts of Twitter may not be working as expected right now. We made an internal change that had some unintended consequences. We’re working on this now and will share an update when it’s fixed.
— Twitter Support (@TwitterSupport) March 6, 2023
"Some part of Twitter may not work as expected at the moment," the company's support account tweeted. "We made internal changes that had some unintentional consequences."
The changes in question are part of the project to close free access to the Twitter API, the Platformer can now confirm. On February 1, the company announced that it will no longer support free access to its API, which effectively terminates the existence of third-party clients and dramatically limits the ability of outside researchers to study the network. The company has built a new paid API for developers working with.
But as a sign of how deep the cuts were made by Elon Musk to the company, only one site reliability engineer was placed on the project, said a source quoted by The Verge. On Monday, engineers created a "bad configuration change" that " effectively damaged the Twitter API," according to current employees.
The changes have consequences that are spreading within the company, bringing a lot of internal Twitter tools along with the APIs visible to the public. On Slack, engineers responded with variations of "garbage" and "Twitter down - everything" as they tried to fix the problem.
Musk is furious, said the source quoted.
"The change of small FIRE had a huge impact," Musk wrote on Twitter at the end of the day after Twitter investor Marc Andreessen posted a screenshot showing the company's API failure trending on the site. "The code site is so fragile for no apparent reason. It will eventually require complete rewriting."
Job cuts without stopping have left the company with less than 550 full-time engineers. Some employees are currently sympathetic to the view, which puts at least some of Twitter's mistakes on the technical failures that occurred before Musk's ownership of the company. Fal whale became an old Twitter icon for some reason.
"There is so much technical debt from Twitter 1, so if you make changes now, everything will be damaged," said an employee today.
However, when Musk took over the company, he promised to dramatically increase the speed and stability of the site. His colleagues filtered out existing staff for their technical skills, eventually cutting thousands of workers who were deemed insufficient "technical" to succeed under Musk's leadership.
However, a relentless cut in employment has left the company with less than 550 full-time engineers, we were told. And as former employees have predicted from the start, this loss has made Twitter increasingly vulnerable to catastrophic outcomes.
The wrong configuration change on Monday was at least the sixth famous service outage on Twitter this year:
On January 23, Android users were temporarily unable to post a new tweet or post it. On February 8, the error message told users that they were "exceeding the daily limit to send Tweets," preventing them from posting. On February 15, tweets stopped publishing. On February 18, the chronology and replies were lost. On March 1, the chronology stopped working.
"This type of disturbance has happened too often, so I think we are all cured of it," said an employee at this time.
And it's just a matter of service disruption. Another problem, such as what causes Musk's tweet to be more visible in the timeline than other users, also shakes up users.
In many ways, the disruption last Monday was Musk's final result in the company's leadership. In his bid to cut costs after buying $44 billion in Twitter, he continued to cut staff and reduce free offerings on Twitter.
This paved the way for one engineer to be placed on a large project - which is linked to several critically connected systems that rely on users and employees.
And with a few experts at restoring services, Twitter takes all morning to fix the problem. "This is what happens when you fire 90 percent of the company's employees," said one Twitter employee.