In December 2020, the Fireblocks’ research team cooperated with a request from Stakehound to create a set of “BLS key shares” for BLS credentials related to an ETH 2.0 staking project. The key shares created in connection with this project were managed outside of the Fireblocks platform and were not part of its MPC production wallet structure or backup procedures. When certain irregularities around the BLS key shares were discovered during a regularly scheduled disaster recovery drill, Fireblocks immediately suspended the potentially impacted addresses and offered its help to the customer. We are actively investigating the situation and assisting all parties involved to resolve the issue.
No Fireblocks production keys were ever affected, and all Fireblocks customers’ funds are safe, and customer keys are backed up and recoverable.
Key management outside of the Fireblocks platform
The Fireblocks platform serves over 400 of the most innovative startups and fintech companies in the world. We help them conceptualize, implement and secure the most groundbreaking ideas that will disrupt financial services for generations to come.
Today, the Fireblocks platform has secured over a half-trillion dollars in digital assets and has become a global industry standard for operating a digital asset business.
This particular ETH 2.0 staking project was managed outside the Fireblocks platform. The client requested the assistance of our research team to create an open-source library to generate a threshold BLS key where:
- The customer did not store the backup with a third-party service provider per our guidelines;
- The BLS key shares for this project were not part of the Fireblocks MPC wallet structure; and,
- The keys were generated by the customer and stored outside the Fireblocks platform.
For customers running their business on the Fireblocks platform that uses MPC key shares, we have automated, multi-tier backup procedures that run on an hourly basis and which are regularly tested during our scheduled disaster recovery procedures. Our customers’ production MPC (ECDSA and EDDSA) keys are backed up, recoverable, and not affected.
Disaster recovery policy
As a non-custodial and direct-custody technology provider, Fireblocks’ policy is to require all customers to engage the disaster recovery services of a third-party service provider or to independently backup their keys. This requirement is communicated to our customers during the procurement process as an explicit written obligation in our standard form licensing agreement, and this expectation is reinforced visually and verbally, during onboarding.
How we discovered the locked ETH
On April 29th, 2021, the Fireblocks team conducted a regularly scheduled disaster recovery drill. During that drill, our team discovered that a set of BLS key shards from the backup could not be decrypted.
While we had no contractual obligation to store part of the BLS keys, Fireblocks received the partial BLS shards as part of a one-off ETH 2.0 staking project. Due to the project’s unique nature, we could not create the BLS key using the Fireblocks MPC system and therefore could not use our MPC production system with its associated backups. We opted to provide a one-time advisory service to the partner for this project, where we briefed them on the procedure to shard and back up the BLS key that was sent to us.
Within a few hours of the disaster recovery drill on April 29th, 2021, we concluded that the customer never backed up the BLS key shards with a 3rd party disaster recovery service, as advised, before they used the keys in production.
The Fireblocks team immediately suspended the option to send ETH to this address on the off chance that the keys become unrecoverable.
Unlocking the ETH staking rewards & restoring funds
The Fireblocks team is diligently working to assist all parties involved in regaining access to the ETH and resolving the loss.
We are currently working on multiple solutions, including:
- Continuous attempts to recover the keys using forensics;
- Cryptanalysis of the key generation library; and
- Proposing a long-term ETH 2.0 solution to remediate such incidents on a holistic basis.
We will continue to provide updates on the status of this incident as new information is available.
