On August 5th 2025, we experienced a 33 minute network disruption that halted block production on Base Chain Mainnet. We want to share what happened, how we addressed the issue, and the steps we are taking to prevent similar incidents in the future.
To ensure high chain uptime and minimize single points of failure, we operate several sequencer instances in a highly available (HA) cluster. The cluster is managed by Conductor, an OP Stack component responsible for the reliability and availability of the system.
At 6:07am UTC, the active sequencer began to fall behind due to onchain activity. Conductor detected this and performed an automated handoff to a new sequencer, a standard procedure designed to maintain chain uptime. However, the new sequencer was in the process of being provisioned and was unable to produce blocks.
Typically if an unhealthy sequencer is elected, it does another handoff. Since Conductor was not yet fully enabled on this sequencer, it was unable to initiate another handoff.
At 6:09am UTC, our monitoring systems detected this issue, and our team was paged to investigate. By 6:12am, an incident was formally declared, and our team began taking steps to resume block building.
Our team took the following steps to resume block building:
By 6:40am UTC, the network was fully recovered, resulting in a total downtime of 33 minutes.
System Improvements:
We will be updating our infrastructure to ensure that when a sequencer is added to the Conductor cluster, it is always able to transfer leadership if elected.
Enhanced Testing:
We will test and deploy these fixes as soon as possible to ensure the robustness of our automated systems.