Exploring Core vs Mainsail: What's Different?

In this blog post, we’ll delve into the significant changes Mainsail introduces and how it distinguishes itself from the current Core version. This blog post does not yet contain information about the EVM and is an overview of pure Core vs Mainsail changes. Let’s dive right in.

#Key Changes in Mainsail

The most notable changes in Mainsail compared to the latest Core version include:

• Introduction of a new BFT consensus mechanism with instant finality.
• Increase in the number of active validators from 51 to 53.
• Implementation of state snapshots to expedite synchronization.
• Removes the necessity for an external database configuration, except for the optional public API.
• Separation of the public API into a separate process for improved scalability.
• Modernization of the codebase with the adoption of ES modules.
• Switch to pnpm as the primary package manager for enhanced dependency management.
• Change in the project’s license from MIT to GPLv3.

These changes collectively enhance the security, scalability, and performance of Mainsail, positioning it as a robust and innovative blockchain solution.

#Technical Comparison

The technical differences between Mainsail and the latest Core version are:

#Project Insights

Let’s go over some of the technical, codebase, and general changes that are going to be in Mainsail.

#Structure & Packages

Mainsail’s code originates from the Core V3 repository . However, there are noticeable changes to the project structure that can be observed at a glance. The project is split up into 72+ packages compared to the 19 packages found in the V3 repository.

Additional packages are a result of splitting the crypto, core-transactions, and api packages.

#Crypto

The crypto package has undergone significant refactoring and was split into separate address, key-pair, hash, and signature packages. Mainsail allows blockchain creators to implement custom crypto packages or use existing implementations from the Mainsail repository.

The following options are available at this point:

• Key-pair: BLS12-381, ECDSA, Ed25519, Schnorr
• Signature: BLS12-381, ECDSA, Ed25519, Schnorr, Schnorr-secp256k1 (deprecated)
• Address: Base58, Bech32, Bech32m, Keccak256, SS58
• Hash: Bcrypto, Noble, Wasm

Note: Some combinations of packages might not work.

#Transactions

The transaction package was split into individual transaction packages. We currently support:

• Transfer
• Multipayment
• Multisignature registration
• Validator registration
• Validator resignation
• Username registration
• Username resignation

New second-signature registrations are no longer possible. Existing second signature-enabled wallets will still be supported.

Note: Core V3 distinguishes between v1 and v2 transactions. Mainsail only supports v2 while labeling it as v1. In other words, a Mainsail v1 transaction is equivalent to a Core v2 transaction, and Core v1 transactions are no longer supported.

#API

The Public API is no longer part of the core process but is now a separate process known as mainsail-api.

The Mainsail node still exposes a single API endpoint for sending transactions found in api-transaction-pool.

Optionally, api-development can be enabled for debugging purposes. It is recommended to keep this API disabled on a public network.

Code changes result in the following new packages:

• api-common: building blocks for the API including common plugins, base server, JSON-RPC, …
• api-http: API endpoints for the Public API.
• api-database: database package for PostgreSQL used by Public API.
• api-sync: sync service that seeds PostgreSQL database from Mainsail process with data (transactions, blocks, wallets, …).
• api-transaction-pool: API on the mainsail node for receiving transactions.
• api-development: development API that allows insight into Mainsail, consensus, state, and ledger.

#Strategy Packages

Other packages have also been split up, and sometimes they can be used to provide different strategies and implementations to Mainsail. As seen from the crypto example, you can build up a Mainsail blockchain using different variants. We provide some of the possible variants, but other developers can also implement their implementations and pack them as plugins.

#Logger

Users can switch from the default pino logger to the winston implementation.

#Fees

Users have the option to select between Static fees (a predefined fee for transactions) and Managed fees (fees based on transaction length). More options will be investigated in the future with the implementation of the EVM.

#Validator Set

The Validator Set is a fundamental building block in Mainsail that provides a list of active validators utilized by consensus and block processing. Active validators represent a subset of registered validators responsible for block creation and signing. Mainsail defaults to using 53 active validators.

Different strategies can determine the active validator set. Mainsail offers 2 options:

• Static: Utilizes genesis validators that cannot be changed. Useful during development phases, but not advisable for use in production environments.
• Vote Weighted: Recalculates the validator set every validator round (53 blocks) based on the vote balance, closely resembling the mechanism in Core V3.

#License

Mainsail is licensed under the GNU General Public License v3.0 .

#Toolchain

The project continues to use TypeScript as its primary language. It’s a pnpm monorepo split into multiple packages, also known as workspaces .

We transitioned from yarn to pnpm package manager due to known issues with yarn on V3, including problems with dependencies using different versions, as well as performance and disk space improvements.

We utilize Lerna as the primary build system for monorepos. The new Lerna version employs NX in the background. Scripts are parallelized when possible to improve build times, resulting in faster build times on systems that utilize multiple processor cores.

To provide browser builds for most Mainsail packages, we ported the entire codebase to ES modules . This enables signing and interaction with Mainsail from a web browser.

#For Developers

If you plan to make changes to the project, develop plugins, or maintain a fork, we recommend using a code editor or IDE with extensions that support Prettier , ESLint , and EditorConfig files.

We recommend Visual Studio Code with the following extensions:

• EditorConfig for VS Code
• ESLint
• Prettier - Code formatter

#Tests

The Uvu test runner has replaced Jest in Mainsail. Uvu is lightweight and offers better performance compared to Jest.

While Jest tends to be an all-in-one framework for testing, uvu is much smaller and focuses specifically on being a test runner. Additional packages are utilized to support tests:

• sinon : Provides test spies, stubs, and mocks.
• nock : Facilitates HTTP server mocking.
• esmock : Offers advanced dependency mocking capabilities.
• c8 : Used for code coverage analysis.

#Consensus

Mainsail comes with a Byzantine Fault Tolerance (BFT) consensus mechanism inspired by Tendermint. You can learn more about its internals and check consensus pseudocode in the article: The latest gossip on BFT consensus .

#Block Finality

The new consensus mechanism in Mainsail introduces block finality with the last block. Once a block is committed, it becomes irreversible. This commitment is acknowledged by the majority of active validators who have seen and confirmed the block.

As a consequence of this feature, rollback becomes impossible. Mainsail removes all logic related to rollback. Transactions that are included in the last block are considered final and cannot be reverted.

Forking at the blockchain level is prevented as long as there are more than 2/3 of honest validators (Byzantine Fault Tolerance) participating in the network.

However, in the rare event of a network fork, resolution can still be achieved through social consensus.

#Block Cycle

A block can only be accepted by the network under certain rules. Active validators participate in the block acceptance process on every cycle. A block is deemed acceptable when more than 2/3 of validators have seen and confirmed it through a process comprising two vote phases and a commit phase.

#Proposal

Each attempt to accept a block at a specific height is termed a round. The round number increases with each attempt to produce a block by a proposer, one of the active validators. On each successful block commit, this round is reset to 0.

Before proposing a new block, there is a minimal delay (default 8000 ms) to allow for enough processing time between blocks. After this delay, the proposer prepares a proposal message containing the new block and broadcasts it.

The proposer is chosen from the list of active validators, and with every round and height change, a new proposer is selected.

#Prevote

Validators that accept a proposal with a valid block broadcast a prevote message containing the blockId. If a block is invalid or not received within the predefined time window, validators broadcast a prevote message with a null value.

#Precommit

The consensus proceeds to the precommit stage when a validator receives more than 2/3 of prevotes. If a validator does not receive sufficient prevotes in time, it broadcasts a precommit message with a null value. Similarly, if a validator receives more than 2/3 of null prevotes, it broadcasts a precommit message with null.

If a validator receives more than 2/3 of prevotes with a valid blockId, it broadcasts a precommit message with the blockId.

#Commit

When a node receives more than 2/3 of precommits with a valid blockId, the block is committed and added to the ledger. The consensus process advances to the next block height.

If a node receives more than 1/3 of null messages, indicating the block’s non-acceptance, a new proposer will provide a block for the next attempt without increasing the height.

#Extra Notes

The next proposer has the option to either re-propose the same block or propose a new block, depending on the node’s lock status. This overview provides a basic understanding of the algorithm, but there are additional complexities such as locks and timeouts that are not covered here.

Timeout values for each stage are defined in milestones (stageTimeout = 2000ms). Additionally, the stage timeout is increased by a predefined delta value (stageTimeoutIncrease = 2000ms) each unsuccessful block attempt (round) to mitigate message delays and address network fragmentation.

#BLS12-381

Validators register BLS12-381 public keys in the validator registration transaction. They utilize BLS12-381 private keys to sign consensus messages, including proposal, prevote, and precommit.

For a more in-depth explanation of how BLS signatures work, refer to this resource .

#Aggregation

The main reason we opted for this type of signatures, and that we want to highlight, is because they offer a special property called aggregation amongst other important advantages over Schnorr. This makes them more suitable for a BFT consensus. Message signatures can be aggregated into a single signature, resulting in a significantly smaller commit size as we store only a single signature instead of 53 signatures.

To verify an aggregated signature, a node can provide a list of public keys from validators that were used in the signature aggregation.

#Database

The primary database used in Mainsail is LMDB , a high-performance key-value store. LMDB stores full blocks by height and block ID.

LMDB offers superior performance and replaces PostgreSQL for all non-API use cases in Mainsail.

The core Mainsail instance doesn’t require additional database setup as known from V3. LMDB is provided as an npm package and utilizes local disk storage.

#State

Since LMDB only stores full blocks, the state is bootstrapped on every boot by reprocessing all blocks.

To expedite this process, the current state is exported to disk every N blocks (1000 by default), which includes wallet data. During bootstrap, the node reads the most recent export it can find and skips reapplying blocks that are already part of the snapshot.

This speeds up node synchronization and bootstrap, preventing slowdowns as the chain progresses.

State refers to an in-memory representation of wallets and consensus-related data. Wallets are objects with different attributes. These attributes hold various kinds of values (boolean, string, custom objects, etc.). Developers can extend wallet or state attributes to support custom implementations. New attributes are automatically handled by the state exporter and importer.

#API (Optional)

The Public API has been moved to a separate process. When optionally enabled, the node process syncs all data via api-sync to a PostgreSQL database on each commit. There is no direct communication between the API and the node process. Because of this, the first time the API is enabled a full sync from 0 is required. The Mainsail node provides data while the API is serving it by reading it from the database.

The Mainsail API aims to maintain compatibility with V3 data models but may add additional/new endpoints as needed.

#Conclusion

In conclusion, Mainsail represents a substantial advancement over Core V3, with notable improvements in consensus algorithm, database management, and modular architecture. These enhancements are designed to elevate performance, scalability, and the overall developer experience. We invite you to join us on this Mainsail development journey, and we hope you’ve gained valuable insights from this overview.

#Follow on Twitter (X)

Follow us on X (formerly Twitter) and keep checking the blog to stay up-to-date on all ARK-related updates. We also post a weekly development report so you can easily see what we’ve been up to and follow along our journey towards making your decentralized future a reality.