Toyota says filled disk storage halted Japan-based factories

Toyota says a recent disruption of operations in Japan-based production plants was caused by its database servers running out of storage space.

On August 29th, it was reported that Toyota had to halt operations on 12 of its 14 Japan-based car assembly plants due to an undefined system malfunction.

As one of the largest automakers in the world, the situation caused production output losses of roughly 13,000 cars daily, threatening to impact exports to the global market.

In a statement released today on Toyota’s Japanese news portal, the company explains that the malfunction occurred during a planned IT systems maintenance event on August 27th, 2023.

The planned maintenance was to organize the data and deletion of fragmented data in a database. However, as the storage was filled to capacity before the completion of the tasks, an error occurred, causing the system to shut down.

This shutdown directly impacted the company’s production ordering system so that no production tasks could be planned and executed.

Toyota explains that its main servers and backup machines operate on the same system. Due to this, both systems faced the same failure, making a switchover impossible, inevitably leading to a halt in factory operations.

The restoration came on August 29th, 2023, when Toyota’s IT team had prepared a larger capacity server to accept the data that was partially transferred two days back.

This allowed Toyota’s engineers to restore the production ordering system and the plants to resume operations.

This incident underscores the intricate nature of IT challenges and the consequences they can have when routine maintenance tasks are not properly planned.

A seemingly minor oversight, like miscalculating necessary disk space for data migration or simply running out of storage space, can result in multimillion-dollar losses for major organizations.

Toyota has had a rough year so far in terms of IT system security and data management.

In May, the carmaker reported twice that system misconfigurations led to potential data leaks impacting millions of customers, exposing their private information over several years.

Earlier in the year, researchers discovered API flaws in Toyota’s apps and security gaps in the firm’s supplier portal, leading to unauthorized access to confidential data.

Original Source