## The Incident

On Saturday, June 29, Kreivo stopped producing blocks after a runtime upgrade to version [0.13.0](https://github.com/virto-network/virto-node/compare/0.12.1...0.13.0). This upgrade aimed to introduce pallet contracts, enable bridged DOT reception from Kusama AssetHub, and update our PolkadotSDK dependency from version 1.5 to 1.12. The main goal was to prepare Kreivo for async backing, which would reduce block times from 12 to 6 seconds.

## The Issue

Despite thorough testing before the upgrade, the problem only appeared hours after enactment(at the end of the era). The current rapid development of the PolkadotSDK makes it challenging for small teams like ours to constantly update and migrate code. Our main issue stemmed from the significant version jump (1.5 to 1.12), likely missing crucial intermediate migrations.

## Impact

While most of our users (DAOs) are aware of Kreivo's beta testing stage and aren't critically affected, two key exceptions are:

1. Virto team: Our salaries are paid through the DAO's governance, which is currently inaccessible.
2. KSM Community: This decentralized voice in Kusama can't vote on OpenGov proposals.

## Proposed Solution

While this proposal is being voted on, we'll prepare an updated WASM of the runtime that fixes the issue. Unlocking Kreivo means the "manager account" will be able to do a `schedule_code_upgrade` to manually set the updated parachain's code in the relay chain to match the code manually set by collators with the `codeSubstitute` feature of the chainspec.

## Request

We ask Kusama governance and the Fellowship to approve unlocking the Kreivo parachain so we can restore functionality. After resolution, we'll conduct a public postmortem and implement additional safeguards for the future.

We are glad this kind of issue occurred this early in the lifetime of the chain when impact is manageable and are now more convinced builders need simpler ways to manage this kind of outages, ironically I believe a "governance parachain" like Kreivo where parachains can host their governance can be a great solution to this issue as people can vote to unlock a parachain even if it has halted.

- **Problem Statement:** Kreivo stopped producing blocks after a PolkadotSDK upgrade, affecting the Virto team's salaries and the KSM Community's voting ability.
- **Significance:** The rapid PolkadotSDK development poses challenges for small teams, impacting essential functions like governance and payments.
- **Proposed Solution:** Unlock Kreivo for a manual code upgrade to resolve the issue and restore functionality.
- **Expected Outcomes:** Approval to unlock Kreivo, conduct a postmortem, and implement safeguards for future incidents.

#413 Unlock para 2281 (Kreivo)

No Active Proposals