Sunday, June 16, 2024
2021-10-05
By xylophone
Tip the author

Bundles Will Take Over Arweave: A Look at the Permaweb’s First L2

…how can we catch up to 22 years of web2 development?

We all know web3 is the future of the internet. The resistance against centralized services and vendor lock-in has been building for years, waiting for the right technology to enable us to overthrow the incumbent platforms of web2.

The average user might not express it in these exact terms, but we want freedom to own our data. We want transparency from the organizations that control the platforms we use. Or, to put it more bluntly, we don’t want to be controlled at all. The web was built upon open standards of equality and interoperability, but quickly became the property of a handful of overlords in their own walled gardens.

Blockchain technology looks like it can deliver the final blow to knock out the centralization of control, but, being so nascent, there are growing pains to address in order to bring us to feature parity with web2. To the same degree that web2’s weakness is trust, its strength is speed and user experience.

Reaching feature parity with web2

Last week, we held a panel with key Arweave developers to talk about how we can bring the same experience web2 users expect over to the permaweb. One of the main topics discussed was bundlers – something we have been monitoring and covering heavily over the past few weeks here at arweave.news.

The reason behind this is because they have already been transforming the face of the Arweave network.

Bundlr has processed 10,000,000+ transactions in its first month compared to Arweave only processing 1,000,000. As Bundlr expands, the network will be able to process several orders of magnitude more.

On top of this, Bundlr has already been adopted by Metaplex, Glass, and Mintbase, as well as large Solana and NEAR NFT marketplaces.

You can get a theoretical introduction to bundlers here, and a practical one here.

However, in this article we’ll be looking at both angles. How can bundlers help bring the Arweave permaweb closer to a web2 experience, why are they necessary, and what will they enable in the future? But first, let’s get an overview of what bundlers are.

Bundlers 101

an easy to use layer 2 aggregation layer for more reliable and cost effective Arweave permastorage!

Let’s be sure we’re getting terminology right first:

  • A bundle is a batch of transactions posted by bundlers in one layer 1 transaction
  • A bundler is a node running Bundlr Network’s services, which guarantee that the bundled transactions it receives will be committed to the permaweb
  • Bundlr Network is the team behind the bundling technology. They will be releasing their own profit-sharing token, BNDL, which will be used in a PoS system to ensure node integrity

Bundled transactions are sent to nodes in Bundlr network which provide a guarantee that every piece of data in the bundle will eventually be mined. Why is this necessary?

For a start, the nature of a decentralized network is that miners are competing to process the most profitable transactions, and in times of high network activity transactions can become orphaned in favor of others. In cases where transactions are orphaned, they collect in the network’s mempool until they are eventually pruned; Arweave will optimistically cache all submitted data up until a certain point, but if it isn’t mined by the time the cache is cleared, it’s deleted forever.

Transactions could also meet this same fate if the gateway or node receiving them hits its rate limit, or if the block difficulty changes and the reward supplied by the user is no longer enough.

In short, there’s a lot that can go wrong when interacting directly with Arweave, and a layer 2 scaling solution like Bundlr is needed to ensure vital data ends up being stored.

Bundled transactions appearing on ViewBlock. Source

To interact with the only bundler node currently running – bundler.arweave.net – you can use arbundles or arkb. Arbundles is a JS library suitable for writing custom scripts and applications that use bundles, and arkb is a CLI tool that makes it easy for non-technical users to upload arbitrarily gargantuan amounts of data to the permaweb without reliability issues.

With the primer out of the way, and it being clear why a L2 for Arweave’s permaweb is necessary, let’s move on to the technical details.

How bundlers work behind the scenes

Bundles as we know them started life as the ANS-104 Standard – a document written by Josh Benaron, detailing how to write multiple ‘DataItems’ (transactions) into one top level transaction. It built on the ANS-102 standard, which now recommends using the newer 104. The motivation behind ANS-104 was to make it possible to delegate the AR storage payment to a third party, write transactions to the network in a batch, and to increase the network’s throughput capacity.

After the spec for bundled data was accepted into Arweave’s official standards, the first node to handle bundled transactions was deployed as a subdomain of the arweave.net gateway, and the arbundles library to interact with it was released, again, by Chief Bundling Officer Josh Benaron.

A bundler node is a server running the Bundlr Network services. (You can become a node, too! Just follow Bundlr’s docs.) A node stakes BNDL tokens to join the network, which disincentivizes bad behavior including both deliberate manipulation of data and unintentional poor performance. 

As stated in the Bundlr documentation, a node runs five services:

  • The NGINX reverse proxy
  • The HTTP API processes
  • The Redis cache
  • The SQL (Postgres) database
  • and the Worker processes

A bundler node captures incoming data and stores it until it can successfully be stored on Arweave. A node can also guarantee what the eventual transaction ID will be when the network accepts the data.

The future of bundlers

Bundlers are most likely going to be the technology that seamlessly brings AR payments for permaweb storage to different chains without any need for end users to bridge, convert tokens, or jump through any hoops beyond paying with their native token.

We’re a few weeks out from seeing the option to host a bundler node which accepts payment for uploads in SOL and converts that SOL to AR in order to buy the user’s desired storage amount from the Arweave network. Theoretically any token could be implemented in this process, circumventing the need for end users to hold AR to interact with the network. Right now, SOL is much more widely available – especially to U.S. customers – than AR.

We’ve also heard from devs at the Bundlr Network that an upcoming feature on the roadmap is support for SmartWeave contracts. Right now, traditional Arweave data transactions are supported but there is no bundler support offered for mass interactions with SWCs.

Along with gateway improvements and other scaling optimizations, bundlers will become part of the solution that rapidly brings web3 up to speed with web2’s UX and functionality. As the Arweave network grows, it’s getting stress tested by heavier and heavier loads, and solutions are being developed to problems we didn’t realize we had. The permaweb is the future, and it’s not too distant.

Join our
Telegram / Discord / Twitter

In Arweave PSCs
Tagged with In No tags

xylophone

Sign up for newsletter

Sign up here to get the latest news and updates delivered directly to your inbox.