The new ARK Core v2 has been running stable on the ARK Public Network for over two weeks, and development isn’t slowing down anytime soon! We will continue improving and implementing new features with input from the community and wider audiences.
Work has already begun on several exciting new features for the ARK Core. With v2 completed, the development team will roll out a much more aggressive release schedule. In this post, we would like to outline some of the upcoming changes and a general outline for the months ahead.
These upgrades will be using standard versioning that adheres to the Semver model:
- MAJOR: 2.0.0 was the initial non-backwards compatible release on the Mainnet.
- MINOR: 2.x.0 will add new functionality and cover bigger changes and new features that introduce higher end impacts (2.1.0, 2.2.0, …).
- PATCH: 2.0.x will be backwards-compatible bug fixes, minor improvements and small additional features (2.0.1, 2.0.11, …).
Although technically following the Semver model, our “Minor” releases are going to be much more than minor. In this post we’ll cover bigger features and changes in the upcoming Core upgrades (please note that the order of integrations may change).
ARK Core 2.1.0: Fully Migrate from JavaScript to TypeScript
The Core development team has already begun work on the full migration of Core v2 from JavaScript to TypeScript. TS allows for static type-checking along with the latest ECMAScript features.
TypeScript is also a much stricter language than JavaScript and lets you spot mistakes that could otherwise go unnoticed. It is more convenient to use and allows for quicker error debugging. For developers, very little will change as the syntax is almost the same as JavaScript (Typescript is a superset of JavaScript). TypeScript makes perfect sense for growing projects that need to scale and have a variety of contributing developers.
We are already in the final stages of moving from JS to TS. Once testing of the new TS implementation on the ARK Developer Network is complete, the ARK team will release the codebase and the Delegates will upgrade the ARK Public Network.
ARK Core 2.2.0: Infrastructure Improvements for AIP 11 & AIP 18
The ARK Core 2.2.0 upgrade will bring with it several infrastructure upgrades that are necessary to for the development of AIPs 11 & 18. This includes increasing the VendorField to 256 bytes to open more use cases and possibilities to take the full advantage of our SmartBridge approach.
We are also going to change the block ID. The new block ID will be stored as a 256 hex id by default (same as we already do with transaction IDs). This is to prevent potential block IDs clashing with ARK’s block height.
On the P2P communication side, we will start to leverage the power of serialization, reducing the network bandwidth and making our chain even leaner. Nodes will no longer communicate via human-readable JSON payload format, instead we will activate serialized data and send serialized buffers of transactions and blocks via the P2P network.
ARK Core 2.3.0: New transaction types (AIP 11 and AIP 18)
AIP 11 — Upgrade of Transaction Protocol One of the biggest, most time consuming and interesting changes for the end users (and businesses) is the integration of new transaction types that will provide additional use cases and tools for the ARK Ecosystem.
There will be a total of 4 new transaction types, with one being upgraded to support bigger data input.
New TX types will be:
- Timelock — This is basically a type of simple smart contract that restricts the spending of an amount of ARK at specified address until a predefined future time or block height is met. This is useful for hash based contracts and payment channels. Timelocks can also be used to lock-up ARK for any reason (prevent spending, security, …).
- Multipayments —To decrease the payload on the blockchain and account for a large volume of transactions, you can combine (batch) payments into multi-payment transactions. This will also decrease the fees per payment. (ie: sending 16 tx in a batched payment at once only incurs one single tx fee instead of 16 different fees.) This will definitely come in handy for delegates that are running daily payouts to numerous voting addresses. At the start we will limit this to 16 possible outputs (16 payments per transaction) and increase by need with further testing.
- Delegate resignation — This gives a delegate the option to resign their position as a delegate by making it impossible for the community to vote for the resigned delegate.
- IPFS — Similar to our SmartBridge Vendorfield, with specific rules on what can be put into the IPFS field (ie: hash of the file). The use case for this is to provide an easy way to timestamp and optionally encrypt and verify files via interface we’ll be building. This implementation of the IPFS tx type won’t allow storing data on the blockchain as for that you need to run special IPFS nodes, but it will provide the tx type to verify data integrity of files via blockchain.
To learn more about AIP-11 click here .
AIP 18 — Multi-signature protocol ARK Core v2 already supports multi-signatures, but the current implementation is lacking. As such, we have decided to implement a much more powerful and useful protocol. We are likely to implement “Simple Schnorr Multi-Signatures” as described in this paper .
Legacy protocol:
- Rule 1 — multisign registration:
ownerPublickey(from where the address is derived), list of committing public keys (N),lifetime(from 1 to 72 hours),minminimum (0 < M < N+1), list of N signatures. - Rule 2 — transaction acceptance: minimum of M different signatures corresponding to different committing public keys should be included into the transaction.
- Rule 3 — if not enough signatures are present, the transaction is stocked into the transaction pool for
lifetimehours until enough signatures are sent via the API (this rule has already been removed from current implementation)
New protocol:
- Lifetime is removed from protocol together with above rule 3.
- Address is derived from a combination of committing public keys, removing the need of the owner’s public key
base58_check(version + ripemd160(sha256(concat(pk1, ..., pkn)))). - Proposed
versionis0x05to match P2SH version on Bitcoin (address starting with 3).
To learn more about AIP-18 click here .
ARK Core 2.4.0: Core CLI and Core app via yarn global
Core will become an npm module that can be installed and interacted with globally. For example you will be able to call ark relay:mainnet after installing core via yarn global add @arkecosystem/core.
Configuration and options will be managed in the same manner via ark configure, which will bring up an interface to manage it. This will essentially be an alternative approach from Ark Core Commander for all who’d like to have everything as part of the Core package to manage their nodes with a few simple commands (eg. ark update, ark relay:restart, ark forger:start, ark relay:monitor, …).
ARK Core 2.5.0: Halfway to v3 — Dropping all legacy API and RPC support (v1)
EOL (End of Life) for v1 API and the old RPC will come halfway through our v2 development cycle. This will likely be around mid 2019, which gives all of the developers that are still relying on the v1 API enough time to migrate to API v2. We’ll announce an exact date of when support will be ending for v1 and RPC once we are closer to finalizing the 2.3.0 release.
If you are still using the old RPC or v1 API please take a look at our documentation for the new API v2 and JSON-RPC:
- API v2 docs:
- JSON-RPC docs:
Getting rid of legacy API code will also further clean our codebase and help with being compliant of the latest API trends and standards.
ARK Core 2.6.0: Rewrite P2P layer to utilize different technology to improve performance and reliability
P2P, or Peer To Peer layer, is where the communication between ARK nodes occurs. It’s the communication level for nodes to interact with each other — passing data back and forth, coming to conclusions based on a predefined set of rules, … (eg. forger node 1: “Hey, I want to include this block in the chain.”, Forger node 2:“OK, all seems to be good, you have my blessing!”).
With API calls there is only one problem — every API request requires a new connection to be opened, and each request adds additional time.
There are a few options here and we’ll be doing extensive research on providing the best overall improvement and performance increase without jeopardizing security.
Currently we are investigating switching from doing P2P via API to P2P via Websockets. As such we are first going to do the POC of the P2P via Websockets and see if its the most viable solution to improve the P2P layer. We’ll do extensive testing and comparisons (all the pros and cons) and if it proves to be the better solution than the current implementation we’ll move it to production.
The advantage of Websockets over API requests is that you can open a persistent connection with another node. Now, you just send data around as you please without the need to open a new connection for each request you make. This not only reduces outside noise, but also provides much faster transfer of data between nodes.
One of the ideas for P2P layer improvements is switching to WebsocketsThis is just one of the ideas currently floating around, and we’ll dive deeper into this once we are closer to this cycle of v2 development.
ARK Core 2.7.0: Implement Parallel Block Downloading
One of the bottlenecks of the current P2P layer over API is also block downloading when you are syncing your node from the network.
At the moment we download blocks in packages of 400, one after another which is two to three times faster than v1, but still takes a lot of time to sync from block 0 to the current height (this can vary depending on a lot of factors — connection speed, hardware, other peers you are downloading from). Currently syncing from zero takes anywhere from 8 hours to 15 hours.
This process can be improved by implementing “parallel downloading”. Connections can be opened with several nodes instead of just one, and large chunks of different blocks can be downloaded simultaneously from different nodes. The receiving node “glues” them together in the appropriate order, much like when downloading a file via torrent where you connect to “seeds” and download different chunks of data of the same end file.
Implementing threaded/parallel downloads will provide another boost in performance and will cut down synchronization time by several hours.
Parallel block downloading will most likely be done via streams as they are one of the most powerful features of the Node.JS.
ARK Core 2.8.0: Configuration presets, hot reloading and easier updates
We are going to provide more preset options when starting up your Core node for things like exchange, forger, API relay, minimal relay (purely a relay, no public API or any other public surfacing endpoints) to provide faster zero-configuration setups that will exponentially help you get up and running depending on your use-case and specifications of your needs.
With this release we’ll also be implementing “hot reloading”. When the configuration is modified, plugins that are affected by the changes will be reloaded at runtime and do not need the Core process to restart.
ARK Core 2.9.0: Implement full public JSON compliant API
The JSON-API will be based on our already improved API v2, but will be extended to be 100% compatible with JSON API specifications.
By following JSON API shared conventions, we will be able to increase productivity, take advantage of generalized tooling and focus on providing a standardized format that can be easily followed by anyone that wants to implement or interact with the ARK blockchain.
JSON API is a format that works with HTTP. It describes how clients should request or edit data from a server, and how the server should respond to specific requests. A main goal of the JSON API is to optimize HTTP requests; both in terms of the number of requests and the size of data packages exchanged between clients and servers; sometimes eliminating network requests entirely.
ARK Core 3.0.0: Plugin system refactor
At the end of the v2 upgrade cycle, additional changes and improvements will be made to the current plugin system.
Currently plugins contain an index.js file that simply exports an object with information about the plugin and how to register it.
The refactored plugin system will provide support for plugin verification and more streamlined development through provided interfaces that need to be satisfied by plugins to guarantee the same implementations throughout the Core.
Wait, where is ArkVM (AVM) in this?
Don’t worry! ArkVM is not part of the Core framework development cycle and will be developed in parallel as an independent project. We will write more on this matter once we have the final specifications and plan fully sorted out.
Epilogue
This sums up our plan for Core in 2019. A lot of Core development plans are going to be developed in parallel. You will be able to track our progress via GitHub projects here (milestones and tasks will be added in the upcoming days):
Please note that we may rearrange milestones. Some upgrades may be done before others and as such be pushed when done. We will try to follow this layout as much as possible. We’ll keep everyone updated if we decide to take a different route at certain stages during development.
