Ethereum’s Shapella Upgrade Went Through Despite ‘Ridiculous Bug’

The latest All Core Developer call highlighted an issue with Ethereum’s leading consensus client, Prysm

article-image

sdecoret/Shutterstock modified by Blockworks

share

Ethereum core developers met for their first group call since the activation of the Shanghai-Capella fork on Thursday to prepare for the next phase of the network’s evolution.

Tim Beiko, who runs protocol meetings for Ethereum, briefly congratulated the assembled devs before getting down to business.

“Great work everyone on Shapella — it worked,” Beiko said.

But there was one issue of consequence: A bug in the main execution client Prysm, related to MEV-Boost, caused validators to fail to propose blocks — a crucial part of the normal functioning of the blockchain.

The Prysm team issued a post-mortem analysis on Friday, identifying the root cause as a missing field during the “unblinding” of the “builder blinded block.”

In the post-Merge Ethereum blockchain, the “builder blinded block” is a type of block that has been processed by a tool called the “builder” to hide certain information. “Unblinding” this block refers to the process of revealing the hidden information using a cryptographic technique. The missing field — BlsToExecutionChanges — was supposed to be populated during this process, but it was not, which caused the issue.

“It’s safe to say that most of the missed blocks were due to this bug,” pseudonymous Prysm developer Potuz told his colleagues on the Thursday call.

This bug was difficult to detect when Shapella was activated on testnets, like Goerli.

Luckily, a “circuit breaker” triggered early on, and relayer teams reacted quickly to temporarily disable Prysm support from most relayers, preventing the issue from having a wider impact on network operation. 

However, the incident highlighted several lessons learned, including the need for more comprehensive testing, and closer collaboration with other teams, such as Flashbots.

“We should set up a way that all of our coding is oriented towards testing the builder,” Potuz said. “This is a ridiculous bug that should not have happened at all. There’s many places where this should have been tested, and all of them failed at the same time. This is impressive.”

A retrospective analysis of the Goerli testnet discovered evidence of the bug there, but the effect was not significant enough in the test environment to attract developer attention, the Prysm team explained on the call.

Around 40% of the Ethereum network nodes run Prysm software, but that percentage has been gradually shrinking as the Ethereum community recognized the need for client diversity. Other clients, Lighthouse, Teku, Nimbus and Lodestar picked up the slack to keep the chain running.

With Shapella safely in the rear view mirror, developers can now turn their full attention to the next major upgrade, known as Cancun-Deneb, or “Dencun” for short. Cancun, Mexico, was the site of the Devcon 3 conference, in 2017, and Deneb is a star in constellation Cygnus, the swan and a vertex of the Summer Triangle.


Get the news in your inbox. Explore Blockworks newsletters:

Tags

Upcoming Events

Old Billingsgate

Mon - Wed, October 13 - 15, 2025

Blockworks’ Digital Asset Summit (DAS) will feature conversations between the builders, allocators, and legislators who will shape the trajectory of the digital asset ecosystem in the US and abroad.

recent research

Research

article-image

Sponsored

Injective is not waiting for the future of finance. It is bringing it directly to us, today.

article-image

Bitcoin has been bullish for nearly 1,000 days

article-image

Robinhood announced that it’s building an L2 and also plans to launch staking for US users

article-image

“We’re not really doing anything controversial,” said co-founder Zak Folkman at Permissionless last week

article-image

Why equities are more stable than in past decades, plus advice from Peter Lynch

article-image

As Permissionless speakers talk on-chain RWA potential, tokenized stock platform Dinari secures FINRA broker-dealer approval