ARK Development Report - December 15, 2023

Welcome to our latest Weekly Development Report, where we spotlight the valiant efforts of our development team. Throughout the week, our attention was directed towards incorporating ARK’s Insights into the Statistics page. Simultaneously, on the Mainsail side, we improved sync performance, along with implementing various other updates.

#Development Summary

Below is a breakdown of total commits and authors by project for development activity over the last week from December 8th through December 15th, 2023.

Overall, the team has demonstrated consistent productivity and engagement over this period, with a total of 39 commits across all projects.

The number of commits and data for each project will fluctuate every week depending on internal sprints, objectives, and difficulty.

#ARK Scan Weekly Progress

Throughout the week, our focus centered on the incorporation of the Insights section into the Statistics page. To provide a more detailed overview of the progress achieved:

• Records were integrated into the data points.
• Address holdings statistics were implemented.
• A section highlighting unique addresses was added.
• Insights into delegate data were incorporated.
• Market data statistics were integrated.
• Developed an annual data statistics section.

“As we progress into the upcoming week, our primary goal is to launch the upgraded Statistics page, a significant milestone for our team. Concurrently, we will focus our energies on refining and optimizing the ARK Scan codebase, ensuring its performance and reliability are top-notch. Furthermore, we’re excited to commence the development of the new Dim theme, with its official release planned for the period following the New Year.

#Mainsail Weekly Progress

This week, our Mainsail team has been working on further improvements to the consensus and various sync performance enhancements. To go into details:

• Graceful application shutdown procedures were implemented, activating process interruption. The async dispose method is called on all registered ServiceProviders, allowing registered packages to safely close connections, and file handlers, and store data to disk. The procedure is forcefully terminated if dispose is not completed within the allotted time or if another process interruption occurs during the shutdown period.
• During node sync, blocks are now stored at the end of the download chunk, enhancing sync performance by reducing disk access frequency. Blocks are stored using a database transaction to ensure safety, while standard block processing still stores every block.
• Consensus messages (proposal, prevote, and pre-commit) for the active height are now stored on dispose, improving both sync and standard processing performance by eliminating the need to access the disk every time a new message is received or formed.
• The Crypto worker package was implemented to create a pool of node workers for executing common crypto methods off the main thread. This is used to aggregate block signatures or validator public keys and to verify signatures, with the number of workers adjustable via CORE_CRYPTO_WORKER_COUNT. Logging on workers is disabled by default but can be enabled through CORE_CRYPTO_WORKER_LOGGING_ENABLED.
• Sync performance was boosted by verifying blocks in chunks, and conducting parallel block signature verification using crypto workers. The number of active workers and CPUs significantly impacts sync performance, and chunks consist of blocks up to the end of the validator round to ensure no changes to the validator set during block processing.
• Blocks are retrieved from the state whenever possible to optimize performance and minimize unnecessary disk hits by looking up the last block in the database.
• Most packages were updated to the latest versions, including a TypeScript update to version 5.3.3, with the target build increased to ES2021. Lerna was also updated to version 8.0.0.
• Various fixes were implemented based on discoveries made on the internal testnet.

Looking ahead to next week, our primary focus will be on code cleanup to streamline and eliminate accumulated technical debt, along with addressing issues identified on the internal testnet.

#Feedback & Feature Requests

If you are using our open-source products and would like to provide feedback or request a feature, please feel free to contact us via the contact pages for the specific product you are using or open an issue on GitHub.

Quick access links to GitHub issues pages:

• Mainsail
• ARK Scan
• ARK Vault

#Follow on Twitter (X)

Follow us on X (Formerly Twitter) and keep checking the blog to stay up-to-date on all of our new releases. We post a weekly development report so you can easily see what we’ve been up to and follow along our journey towards making your decentralized future a reality.