ARK Core v2.3: Generic Transaction Interface, Bigger SmartBridge Field & Full SHA256 Block IDs

One of the major changes in ARK Core v2.3 is the introduction of the Generic Transaction Interface (GTI). The first iteration of this interface will make the ARK Core even more versatile! GTI is a prerequisite for upcoming developments including the much anticipated new transaction types — multisignatures, timelocks, multipayments and more. With that in mind, we are proud to present the newest Core release v2.3 completing yet another major milestone of development at ARK.

Only a month has passed since our latest Core release (v2.2), which featured an all new CLI, Core as an NPM package and much more. With v2.3 we are introducing more foundational pieces to the ARK Core that will open up exciting new possibilities for future development. Let’s talk about what’s inside!

#What’s New in Core v2.3?

There are 3 major components of this newest release. Each one makes Core stronger, provides additional functionality and streamlines our development process moving forward. With the ability to create new, standardized transaction types, v2.3 makes ARK more extensible and easier to modify for our bridgechain community.

#1. AIP 29: Generic Transaction Interface

The crowning jewel of v2.3 is the introduction of AIP 29 , Generic Transaction Interface. Thisis a significant change that enhances the immediate use-case utility of the ARK Core. GTI is, in short, a framework to easily implement new types of transactions without the need to get your hands dirty going down into the basement of the Core.

GTI aims to provide developers and businesses an easy way to quickly develop new transaction types with custom business logic behind it. The initial implementation is a technical preview of what comes with our first new custom tx types defined in AIP 11 and AIP 18 coming out soon after P2P WebSockets in Core v2.4. At that point we will reach full functionality of what AIP 29 really aims to be — a template engine for the development of customized transaction types. Custom transaction templates will also be developed for the different segments we aim to tackle — IoT, Governance, and Gaming.

#2. Increased Vendorfield Size (64 to 255 Bytes)

The second change is an increase in how much data our Smartbridge (Vendorfield) can hold. A four fold increase in size will expand the Vendorfield from 64 bytes to 255 bytes. This expansion will provide valuable additional transaction data capacity to expand the use-case landscape providing more room for instructions or other information. This will also provide a foundation for some of the potential features of the new tx types we’ll introduce with AIP 11. The upgrade will be turned on at block height 8,128,000 which should happen around Thursday, the 25th of April 2019 at 12:30:00 UTC.

#3. Full SHA256 Block ID’s

The third major change is the transition to full SHA256 block IDs, encoded as a hex string. In simple terms, this eliminates a potential collision of block IDs with blockchain height or other block IDs, making ARK safer and future-proof. An example of the old implementation of a block ID would be a set of numbers such as “1051922236637126565” which although unlikely, could possibly be generated again compared to using full SHA256 for block IDs like “ad0da204d62191c3bf65f55762ac179227f717a06a42ebbeb29d5539068d9469”. This change will be turned on at block height 8,204,000 which should happen around Thursday May 2nd at 13:30:00 UTC.

For node operators of the current ARK Core, in order to update, please follow these migration instructions (v2.2 to v2.3): https://docs.ark.io/releases/v2.3/migrating_2.2_2.3.html

Some other changes in v2.3 include:

• Core Tester CLI has been fully rewritten from scratch. This toolset helps developers quickly generate transactions, send transactions, generate addresses, and more.
• Winston logger has been replaced with Pino which significantly improves the performance of logging when it generates a lot of data in situations like syncing. Pino is also a much more light-weight integration than Winston, essentially reducing the bloat of huge dependencies in Core.
• As part of AIP29, the transaction asset field needs to be stored directly in the transactions table. A nice side effect is that the integrity verifier can now directly query transaction assets without having to deserialize which greatly speeds the node startup. The initial migration may take up to a few minutes since it needs to populate the asset column for all existing records.
• Added 5 new events to make some potentially hard to grab data more accessible: block.received — generated when a new block is incoming block.disregarded — generated when a new block is disregarded transaction.pool.added — generated when transactions are added to the pool, always generates an array of transactions. transaction.pool.not-added — generated when transactions are not added to the pool, always generates an array of transactions. transaction.pool.removed — generated when a transaction is removed by its ID, only emits the ID of the transaction.
• Proper validation that recipient’s address is on the correct network, mitigating sending to wrong addresses on other blockchains (BTC, ETH, …).
• Default slip44 and wif of the network are now also available via node configuration API endpoint.
• The number of active delegates can now be adjusted via milestones and Core adapts the round sizes dynamically. For example a bridgechain could now start out with 5 delegates and switch to 11 after the 10,000th block.
• Axios implementation has been replaced with Got , which is a much lighter weight library. Axios also has known issues with timeouts that can randomly cause issues and as well maintained as Got.
• Core Webhook integration of SQLite3 has been replaced with lowdb significantly reducing the size of the package and its dependencies.
• Wallets tables have been removed from the DB since they have not been used since the release of Core v2 (it was a relic from Core v1). This will reduce the size of the ARK blockchain and make it more lean, removing obsolete data that is accessible by other means. All wallet related information is served from in-memory which is much quicker and gives you instant real-time data.
• Replaced micromatch (182 dependencies, 4.17M) with nanomatch(11 dependencies, 0.27M) for better overall performance and reduce in dependency numbers along with size of them.

More changes and fixes in Core v2.3 can be found later down the blog post.

#What’s Next for Core?

The next iteration of the ARK Core v2.4 will dig deep into the P2P (Peer 2 Peer) mechanics of how nodes communicate with each other. The current HTTP API will be replaced with WebSockets using the SocketCluster library for P2P communication.

WebSockets make more sense for P2P communication between nodes since they enable active connections to each peer instead of independent HTTP requests. Data transfer time will be improved since no HTTP headers means less data to transfer. Also, it will scale easily by using concurrent connections.

SocketCluster is a solid WebSocket framework offering great features including auto reconnect, middleware logic and multi-threading.

Core v2.4 will also feature much refactoring and cleaning up of dead code. This will make it leaner, cleaner and increase performance.

Core v2.4 testing on Development Network will begin in early May and in less than a month we will have a new P2P layer.

Most will want to hear about how AIP 11 and AIP 18 are progressing. We are heavily into AIP 11 and AIP 18 development and we estimate both will be ready in Q2 of 2019 for testing on Development Network. This will enable even more new features and potential use-cases of ARK. We are excited to get this out as soon as possible!

#Changes In Numbers (from 2.2 to 2.3)

• 14 different developers contributing to the Core.
• 185 new commits to the Core.
• 1,198 files changed in the Core.
• 40,894 code additions to the Core.
• 38,593 code deletions in the Core.

#Breaking Changes

• Removed the wallets table from the database (#2209 )*
• Replace SQLite3 with lowdb in core-webhooks(#2124 )
• Replaced core-logger-winston with core-logger-pino (#2134 )
• Rewrote core-tester-cli from scratch (#2133 )
• Merged core-debugger-cli into core-tester-cli and deprecated it (#2133 )
• Use the node.js EventEmitter from events instead of eventemitter3 (#2329 )

*Note: Core 2.0 has been fully reliant on in-memory wallets since the 2.0 release. This release only removes the dumping of wallets into the database as it is wasted space and doesn’t serve any purpose. If you have applications that rely on the database you should migrate them as soon as possible to using the API as only that data is provided in real-time.

#Added

• Implement AIP29 (#2122 )
• Search delegates by their username in core-api (#2143 )
• Implemented the ark reinstall command in core (#2192 )
• Added the --force flag to the ark update command in core (#2190 )
• Added more parameters for delegate searches in core-api (#2184 )
• Add the /v2/delegates/active endpoint to core-api (#2205 )
• Added restart flags to the ark update command in core (#2218 )
• Added the make:block command to core-tester-cli to create blocks (#2221 )
• Transaction asset column added to the database (#2236 )
• Added the core-error-tracker-rollbar package (#2287 )
• Added the core-error-tracker-raygun package (#2288 )
• Added the core-error-tracker-airbrake package (#2289 )
• Added the core-logger-signale package (#2343 )
• Added more events for blocks and the transaction pool (#2321 )
• Return slip44 and wif via v2/node/configuration (#2388 )

#Fixed

• Properly sort peers by their version (#2229 )
• Memory leak in the monitoring process of core-forger (#2341 )
• Handle dynamic round sizes with milestones (#2370 )
• Validate that a transaction recipient is on the same network (#2394 )
• Handle empty rows in mapBlocksToTransactions (#2404 )
• Prevent indexing/creating of ghost wallets (#2405 )
• Refuse transactions from senders with pending second signature registrations and do not rollback when refusing a block (#2458 )

#Changed

• Increased the vendor field length to 255 bytes (#2159 )
• Replaced micromatch with nanomatch to improve performance (#2165 )
• Replaced axios with got to resolve known timeout issues with axios (#2203 )
• Switch block id to full SHA256 (#2156 )

#Removed

• Removed fast rebuild flag code that hasn’t been used since 2.0 release (#2210 )

#Special Thanks

Thanks to all who have helped test the newest Core on Development Network, prior to moving it to Public Network, by providing valuable feedback, reporting issues, and helping to resolve issues by opening pull-requests.

For those running your own servers with ARK Core, if you experience any errors or issues, please open a Github issue report for easier tracking and resolution: https://github.com/ArkEcosystem/core/issues .