Arweave 2.8.0: Hard Fork on November 13 – What’s New?

Author: Kyle

Translator: Kyle

Reviewer: Marshal Orange

Source: Content Guild - Translation

Arweave 2.8.0 will activate on November 13, 2024, at block height 1,547,120. The update brings a composite packaging mechanism to reduce dependence on high-end equipment, improve mining flexibility and reduce power consumption; the block header structure is optimized to be more adaptable; the oracle makes the price more transparent and stable, and the reference period is extended to 3 months. Additionally, the new version enhances node synchronization speed and database performance, minimizing resource use and maintenance costs, providing users and miners with a faster and more stable experience.

During the Arweave Asia 2024 conference on September 19 in Singapore, Arweave/AO founder Sam Williams previewed the upcoming Arweave 2.8.0 update. The official documentation was released on October 17, detailing the changes.

The hard fork will occur on November 13, 2024, at 14:00 UTC at block height 1,547,120. This update includes composite packing, block header restructuring, pricing mechanism improvements, and performance optimizations aimed at improving network efficiency, reducing resource consumption, and enhancing stability. Below, we explore the key highlights of the update.

Key Features of Arweave 2.8.0

1. Composite Packing – A More Efficient Mining Method

Arweave 2.8.0 introduces Composite Packing, which allows miners to access datasets at a slower rate, thereby enabling the use of larger-capacity hard drives with the same bandwidth, reducing the need for high-performance storage devices. This new system enables a more flexible and precise approach to mining:

• Multi-level packing difficulty (1-32): Miners can choose from difficulty levels between 1 and 32. Higher difficulty levels take longer to pack data but proportionally reduce the read requirements during mining.

• Chunking and hashing: Each composite-packed chunk is divided into 32 sub-chunks, with each sub-chunk being incrementally processed using the RandomX packing function. The number of RandomX rounds increases with the difficulty level. All storage modules must follow a unified packing format to ensure fairness across difficulty levels and prevent exploits.

This mechanism allows miners to adjust their strategy based on their available storage and energy resources, prolonging disk lifespan and reducing energy consumption, leading to more flexible and efficient mining.

2. Block Header Optimization – A More Flexible Data Structure

To align with the new packing mechanism, Arweave 2.8.0 also optimizes the block header structure for better compatibility with various packing formats and improved storage efficiency. Key changes include:

• New field - packing_difficulty: Records the packing difficulty of chunks used in the block solution. This field, together with reward_address, is required to unpack and validate the solution chunk.

• Optimize poa1 and poa2 fields: Adjustments were made to accommodate both the previous spora_2_6 format and the new composite packing format. Previous formats retain full data, while the new format reduces redundancy, storing only essential packaging information.

• New field - unpacked_chunk_hash: Stores hash values for unpacked data to ensure data integrity and prevent tampering.

3. Pricing Mechanism Optimization – Smarter, More Transparent Pricing

Arweave 2.8.0 introduces improvements to its storage pricing mechanism, promoting greater decentralization in price management. Key changes include:

• Introduction of a trustless oracle system: From block height 1,551,470, upload prices will be calculated using a trustless oracle system, with the historical reference period extended from 1 month to 3 months. This change reduces human intervention, making pricing more transparent and stable.

• Extended historical data reference period: Storage prices will now be based on 3 months of historical data (up from 1 month), minimizing the impact of short-term market fluctuations and ensuring more stable and predictable pricing.

4. Performance Improvements and Bug Fixes – Faster Synchronization and Greater Stability

Several optimizations have been made to the database and node synchronization processes, significantly improving system efficiency and reliability while addressing known issues:

• Database optimization: The core RocksDB database has been enhanced to resolve corruption issues, and the repair_rocksdb feature has been removed, improving database stability.

• Improved blockchain synchronization speed: Adjustments to block and transaction header synchronization have significantly reduced the time needed for new nodes to sync with the network.

• API bug fix: The /data_sync_record endpoint has been fixed to prevent it from returning deleted block data, avoiding data confusion and false reports.

What Does “≥75% Lower Resource Utilization” Mean?

Although it is difficult to independently verify the exact percentage reduction in resource consumption, the following improvements highlight how the update can effectively reduce resource usage:

• High-difficulty packing reduces disk reads: Higher packaging difficulty lowers the frequency of data reads, extending disk lifespan and reducing equipment depreciation and energy costs.

• Optimized synchronization minimizes runtime: The improved synchronization mechanism shortens the time nodes need to run, reducing overall energy consumption.

• RandomX algorithm boosts computing efficiency: The new version requires 3,500 HugePages (approximately 7 GB of RAM) when using the randomx_large_pages option, compared to the 1,000 HugePages required in earlier versions. This increase optimizes memory access speed and enhances mining performance.

In summary, the reduced resource usage primarily results from decreased storage load, shorter node runtimes, and improved CPU and memory efficiency.

Conclusion

With over 6 years of continuous operation and numerous version upgrades, Arweave has evolved into a stable and reliable decentralized storage network. However, as market demands and user scenarios evolve, the network must continually optimize. The Arweave 2.8.0 update not only enhances user experience but also provides miners with more flexibility.

For Users:

• Fairer pricing: The oracle system and extended historical data references ensure more transparent and stable prices.

• Faster synchronization: Nodes can connect to the network more quickly, reducing wait times and improving the overall experience.

For Miners:

• Lower device requirements: High-capacity disks can now participate effectively, reducing equipment wear and energy costs.

• Multi-level difficulty options: Miners can tailor their strategies based on available resources, adapting to different conditions.

• Increased stability: Faster synchronization and improved database reliability lower maintenance costs.

• New hardware requirements: The RandomX algorithm requires 3,500 HugePages (approximately 7 GB of RAM), a 2,500-page increase from earlier versions, so miners should prepare in advance.

• Time-sensitive preparation: Miners need to get ready before the hard fork at block height 1,547,120 (November 13, 2024), as mining can only begin after the fork activates.

Each Arweave upgrade moves the network closer to greater efficiency and decentralization. As the ecosystem continues to grow and new features are introduced, Arweave will remain responsive to the needs of both users and miners, ensuring the network’s long-term vitality and resilience.