The onlineization of offline life and the booming owner economy has spawned huge data storage needs, with huge commercial and application prospects;
Due to high cost, preset storage format and other factors, it is difficult for centralized storage to keep up with the changes in storage capacity, data read and write speed, security, and data relationship brought about by the exponential growth of data. Therefore, decentralized storage has emerged as the times require pregnancy;
Decentralized storage is based on blockchain technology and has the characteristics of strong scalability, security, high efficiency, automatic fault tolerance, high reliability, and lower cost;
In the world of Web 3.0, users’ massive online footprints and created online works have created an incredibly huge data kingdom, and the maintenance of this data kingdom depends on a reliable, stable and secure data storage network;
In the world of Meta Universe Huihong, if a monetized virtual item loses the protection of its storage on the trusted blockchain, once its corresponding metadata and media data become invalid or tampered with, it will become an unacceptable one. Party’s worthless cheque;
The current main decentralized storage solutions are the IPFS/Filecoin and Arweave projects. IPFS pioneered the introduction of the incentive layer Filecoin to ensure the reliable storage of data within the agreed period, and Arweave through technological innovation to achieve permanent storage of data on the chain .
The summer of 2021, a group of irregular villain of ten thousand pixels but it looks very strange suddenly rise, surprising is that it was able to single picture is equivalent to tens of millions of dollars in Ethernet Square turnover; thereafter Bored Ape Yacht Club and Loot have become the new favorites of the crypto market. From encrypted art and games to avatars and texts, the rise of NFT has ignited people’s endless imagination of the meta universe and Web 3.0.
However, when we are talking about CryptoPunk as a status symbol, enthusiastically playing and earning the game Axie Infinity, and heartily conceiving the journey of the stars and the sea in the meta-universe, have we ever wondered where these data will be stored? Are you worried that these data may be tampered with? No one hopes that the avatar that has spent tens of millions of dollars is worthless because of the tampering of the underlying data, and no one hopes that the precious memories of talking with friends in the metaverse will disappear forever due to system downtime. Standing at the starting point of the meta universe and the Web 3.0 era, in today’s explosive growth of data, we urgently need a safe, open, free, and reliable storage network.
This article will introduce the profound significance of storage in Web 3.0 and meta-universe, and try to describe to readers the exciting applications of decentralized storage in Web 3.0 and meta-universe. At the same time, this article will also study and compare different decentralized storage solutions based on the development status of decentralized storage.
Industrial ecology of storage
The spread of the new crown epidemic has accelerated the blurring of the boundary between the real world and the virtual world. People’s work, social interaction, entertainment, dining, shopping, travel and other activities in the real world can almost all be performed in the Internet world; at the same time, from Web 1.0 to With Web 3.0, from passive data reading to enthusiastic content creation, data production has exploded and will continue to grow exponentially. According to a 360 research report, the global digital content creation market is expected to reach a market value of $16.9 billion in 2025. The booming owner economic model and exponential growth of data have created huge data storage needs. The development of chip technology, algorithms, hardware, etc. has promoted the advancement of storage technology, and the release of data value and the rigid demand for data security have forced storage. Technological innovation and infrastructure construction. In the iteration and integration of software and algorithms, the storage industry has gradually evolved from traditional disk storage to cloud storage, breaking through the physical limitations of hardware. As one of the three pillars of blockchain technology (computation, storage, network), storage is of great significance to the development of Web 3.0 and Metaverse-from an application point of view, data security and the ultimate ownership of data sovereignty are related The stable operation of the virtual world; and from an economic point of view, the massive data storage demand generated by the meta universe and the Web 3.0 world also has huge commercial prospects.
Global data generation trend chart, source: http://www.slideshare.net/PerryLea/the-20-rule-how-the-seismic-growth-of-data-has-always-and-will-always-outgrow-telcom
Due to various factors, we cannot currently assess the size of the storage market as a whole. However, taking cloud storage as an example, it is estimated that by 2027, the global cloud storage market will reach USD 222.2 billion, with a compound annual growth rate of 21.9% during the forecast period ( Sneha Korad Rachita Rake & Vineet Kumar, 2021 ) .
Global cloud storage industry market size forecast, source https://www.alliedmarketresearch.com/cloud-storage-market
The storage business model is divided into centralized and decentralized storage. Centralized storage is the complete storage of data on a centralized server, while decentralized storage uses distributed storage technology to store data slices in multiple independent locations. Storage provider. Centralized storage solutions meet the needs of enterprise-level storage with high stable performance and low cost; decentralized storage meets the long-tail enterprise market with its customized storage solutions, economic incentive models, and strong privacy. Special scene requirements. Standing at the intersection of Web 3.0 and Metaverse, the importance of data security and user data ownership, the importance of decentralized storage will become more prominent.
2.1 Centralized storage
Centralized storage, that is, the entire storage is concentrated on multiple sets of devices in a system. It is a product of the past mainframe era. It uses a limited fixed number of nodes. Data access only needs to go through a controller, so it has the advantage of low latency. Relatively high costs, data security, and low operational scalability of service providers.
One of the keys to the smooth operation of a centralized storage system is the stable operation of the storage server. The storage server has become the bottleneck of system performance and the focus of reliability. It puts extremely high requirements on the storage environment and hardware equipment, and thus a huge centralized storage market But it is a giant’s cake. The four giants Amazon, Microsoft, Google, and Alibaba Cloud together account for 67% of the cloud storage market.
2021 Q1 Global cloud storage provider market share (picture source: https://www.statista.com/chart/18819/worldwide-market-share-of-leading-cloud-infrastructure-service-providers/)
With the spread of Weibo, Facebook, etc. and the popularity of self-content creation, we have entered an era of exponential data growth. Explosive data growth places higher requirements on storage capacity, expansion speed, and data backup. ; Real-time information interaction between users requires faster data read and write speeds and higher security; at the same time, as the types of data generated by users’ self-creation and the relationship between data become more complex and diversified, the storage system needs to be single from the past. The file types are extended to various semi-structured and unstructured data relationships. Due to its high construction costs and industry entry barriers, centralized storage has difficulty keeping up with the pace of data production due to the expansion of data storage capacity and data types, and decentralized storage has emerged as the times require.
2.2 Decentralized storage
As Hu Feitong, an early preacher of IPFS (InterPlanetary File System), said: “In the world of centralized data storage, users not only pay for data storage, but also do not get good data protection. This phenomenon must be changed. . Through decentralization, users can manage their own data, encrypted storage, authorized access, secure backup and other features will greatly enhance user privacy, and can also enhance the long-term storage of data. At the same time, it is expected to adopt a shared mode Greatly reduce costs. This is the inevitable direction of data storage development.”
Decentralized storage is based on blockchain technology, uses open source applications and algorithms, and distributes data slices to multiple independent network nodes. It advocates privacy protection, data redundancy backup, etc., and uploads files to seed nodes or files. The party provides economic incentives to value data. Among them, the economic incentive model that allows user data to be stored safely and for a long time is the finishing touch to decentralized storage.
When decentralized storage meets blockchain technology, it has the characteristics of strong scalability, security, high efficiency, automatic fault tolerance, high reliability, and lower cost (Jiao Shike, 2020) :
- Scalability : The number of storage nodes can be expanded indefinitely, and the storage capacity of each node can also be flexibly adjusted;
- Security: There is no need to understand the partner or seek the trust of a third party. Stored information is divided into fragments and stored in multiple nodes. The encryption of stored data is not limited to users and software terminals, but also in all links of the storage network. Encryption processing is performed through methods such as private network access keys and zero-knowledge proofs during transmission;
- High efficiency: nodes on the same network can directly share files;
- Automatic fault tolerance: Since the storage content has been distributed to many nodes in the network, additional copies can be stored when data transmission or storage errors occur;
- High reliability: The verification mechanism contained in the storage system can ensure that the files are stored completely and truthfully, and can be accessed by users at any time 7*24 hours;
- Low cost: It is reported that if you store a video file of “Hip Hop in China” at a cost of US$0.001/GB bandwidth, iQiyi has to pay a traditional storage provider US$1 million. If IPFS distributed transmission is used, it can save 60 % Of bandwidth cost.
Decentralized storage: the intersection of Web 3.0 and meta-universe
3.1 Decentralized storage and Web 3.0
The origin of the Internet can be traced back to the birth of the World Wide Web in 1989. If portals opened the Web 1.0 era, then the rise of blogs in 2003 marked the beginning of Web 2.0. From Web 1.0 to Web 2.0, from passive information reception to self-created content and online interaction, Internet users call for an application experience that pays more attention to user content creation, ease of use, and interoperability, and Web 3.0 is about to emerge. As Tim Berners Lee said, “People have been asking what Web 3.0 is. I think maybe when you have a scalable vector graphics overlay on Web 2.0-everything is rippling and folding , It seems vague-and you can access a semantic network integrated in a huge data space, you can access an incredible data resource” (Victoria Shannon, 2006) . Although we have not yet reached a consensus on the definition of Web 3.0, Web 3.0 should have the following characteristics (Fabric Venture, 2020) :
- Open : Open source software is built by an open and accessible developer community, and operations performed on the chain can be viewed by everyone.
- Trustless : The network itself allows participants to interact publicly or privately without a trusted third party. Web 3.0 uses a decentralized incentive mechanism to ensure that participants abide by the rules. In transactions, the consensus mechanism of the blockchain protocol is used to verify the authenticity of the account and the effectiveness of the transfer, and achieve a disintermediation of value transfer.
- Permissionless : Anyone, including users and suppliers, can participate without the authorization of the regulatory agency.
The evolution of the Internet (Source: https://hackernoon.com/web-3-0-will-be-powered-by-blockchain-technology-stack-626ce3f828c7)
In the sea of stars of Web 3.0, the boundary between the digital world and the physical world is blurring. The massive online footprints of users and the online works created by users, if there is not a stable and reliable data storage network to silently record behind the scenes, when we look back at what happened in the Web 3.0 world with great interest, then there may only be left. The next fuzzy fragment.
3. 2 Decentralized storage and meta-universe
In 2021, with the listing of Roblox, the “Metaverse” concept suddenly swept the Internet, VR/AR, and financial investment circles. With the rise of microcomputers, VR equipment and high-speed network technology represented by 5G, people Came to the door of the meta universe era. As our last hotspot research report “Metaverse Perspective: Looking at the Future from the Past” said: The finishing touch of the metaverse world is its economic system. First, the open payment and clearing system based on the blockchain can meet the real-time, fair, transparent and fast P2P payment needs of users in the metaverse world ; monetize the virtual items or innovations of users in the metaverse world , not only It gives users the protection of rights and interests similar to the real physical world, which can greatly promote the circulation and transactions of Meta universe assets and encourage users to obtain wealth accumulation through continuous innovation. Behind this, in the world of Metaverse Huihong, if the huge user information interaction and the created virtual item information loses the protection stored on the trusted blockchain, once the corresponding metadata and media data become invalid or be Tampering, then the NFT ownership certificate in the user’s hand will become a worthless check without an acceptor.
Virtual assets in The Sandbox game, data source: The Sandbox
3. 3 Main decentralized storage solutions
The earliest decentralized transmission solution was Bittorrent, or BT download, which was proposed by software engineer Bram Cohen in 2003. The P2P download method overcomes the limitations of traditional download methods—it does not require resource publishers to have high-performance servers. Transfer resources, and the more people who download the same file, the faster the download of the file, and its free use of the charging model has attracted many netizens to use it. However, Bittorrent downloads must use a torrent file, that is, all addresses of the downloaded content can be downloaded in this torrent file, and the client can only download the content in the torrent file. However, the torrent file may decrease or even disappear over time, which also means that the stored content may disappear while spreading. Another obvious problem with Bittorrent is the lack of incentives and the lack of motivation to encourage users to share files for free. In the absence of appropriate economic incentives, only those widely popular and constantly downloaded content can survive, and some of the essence that has failed to attract public attention may disappear in the long river of history, leaving only a vague silhouette.
3.3.2 IPFS/ Filecoin
IFPS (InterPlanetary File System) is another decentralized storage solution proposed in 2015, which overcomes the shortcomings of bittorrent-it can store multiple versions of content and pre-download content, thereby achieving decentralization, speed, high efficiency, Reliable and safe content storage. It is worth noting that IPFS has pioneered the introduction of Filecoin, an incentive layer, through the token incentive mechanism to ensure reliable storage of files within the agreed period.
Filecoin allows any individual to use free hard disk space and bandwidth to provide data storage and retrieval services on the open market, monetize unused hard disk capacity, and use PoRep (Proof of Replication) and Post (Proof of Time and Space) to ensure that miners actually store files Circumstances, to prevent attacks such as witches, generation, outsourcing and other false stored data. To put it simply, RoRep proves that the miner actually stores the data, while Post proves that the miner has always stored the data in his hard disk. The principle of operation is as follows: Users find suitable storage miners by issuing orders, and the data is encrypted and divided and sent to the miners. The miners add the information to the storage block for storage and continue to prove their work to obtain rewards.
IPFS working principle diagram, picture source: https://www.8btc.com/article/613165
Of course, this model of filecoin does not mean that it is perfect. IPFS mining requires professional equipment-GPU mining machines, which makes mining costs high. The increase in industry entry barriers may cause ordinary individuals to be unable to directly participate in storage mining, and the total storage of the entire network will be transferred to a small number of storage providers or large-scale mining pools. concentrated. To some extent, the complete decentralization conceived by IPFS has become a “centralized” storage monopolized by a few vendors during the execution process.
The Filecoin mainnet was launched in October 2020 and currently has 3,175 active nodes, mainly in North America, Europe, China, South Korea, etc. As of October 14, 21, Filecoin’s entire network has provided 12.393 EiB effective computing power. The current basic rate is 0.1052 nanoFIL, and 32G storage costs 4.958 Fil/TiB.
Filecoin node distribution map, picture source: filscan.io/statistics/map
Filecoin baseline and computing power trend chart, image source: filscan.io/statistics/map
With the improvement of filecoin’s underlying network and infrastructure, the early dividends of its network effect development have attracted more applications to deploy. Up to now, filecoin has established an ecological environment covering the fields of “consumer storage applications, data markets, data retrieval, NFT, Defi, verifiable storage and privacy”, such as Fileswan, NFT Stororage, Opera, Livepeer and other projects.
Filecoin ecological map, picture source: https://github.com/filecoin-project/community/discussions/194
Unlike Filecoin, which stores data off-chain, another storage solution, Arweave, stores data directly in the blockchain. It is a decentralized storage solution that focuses on permanent storage + on-chain storage. As shown in the figure below, the Arweave solution is divided into two layers: the underlying block network Blockweave and the upper layer Permaweb based on HTTP. The technical highlight of the Arweave project is its underlying Blockweave network:
The project uses a new blockchain network Blockweave-based on the consensus mechanism and specific block generation mode, that is, the hash value of all previous blocks is made into a hash list and all active wallets in the system are made into a list. The verification is completed by retrieving the hash list and the wallet list, instead of verifying the entire node like BTC , ETH and other nodes. This method reduces energy consumption and improves the efficiency of the node.
- Incentive algorithm Wildfire:
Unlike Filecoin, which needs to continuously charge storage fees, Arweave only charges a one-time permanent storage fee when data is written. After that, users read data for free. In order to encourage nodes to respond to data read requests, Arweave introduced Incentive algorithm widfire. Simply put, Wildfire is a node scoring system that allows each node to rank other nodes based on the number of new transactions and blocks sent by the node and the speed of responding to requests. The more reliable data obtained from neighboring nodes, the higher the score for the other party, and the greater the probability of block generation.
Smart Weave transfers the computational burden of smart contract execution from the node to the user, that is, only all the data of the application is stored in the Arweave network. These data will be provided to the browser or other client applications in the form of code, thus enabling Arweave to Supports Dapps that require a lot of calculations.
Although the Arweave storage solution saves data on the chain, it can guarantee the non-tampering, “permanent storage” and public access of the data; however, this model also has certain drawbacks. First, the data cannot be tampered with, so that any changes can only be made by uploading new data, which will cause a certain iteration cost. Second, in order to ensure the permanent storage of data, Arweave needs to continuously attract new nodes to join while ensuring that the marginal revenue of nodes is greater than the marginal cost. However, the number of Arweave nodes has been decreasing. There are currently only 146 nodes, compared with May 2020. 490 nodes are reduced by 70%.
Arweave node distribution map, source: https://viewblock.io/arweave/stats
The Arweave mainnet was launched on January 29, 2019, and the Arweave 2.0 version with the bundled transaction storage function was launched in May 2020. The on-chain transaction volume began to work in 2021 and reached its peak in September this year (September The average transaction volume exceeds 80,000 transactions). The launch of Metaplex Candy Machine on Solana and the launch of ipfs2arweave cross-chain bridge have created Arweave’s brilliant performance.
Arweave transaction volume statistics, source: https://viewblock.io/arweave/stat/txCountHistory
Looking at the Arweave ecological project, the most active ones are NFT, DeFi and infrastructure track, such as Koii, Limestone and other projects. The ecology is still in the early stage of construction.
Arweave ecological List (Source: chain smell)
Standing at the intersection of Web 3.0 and Metaverse, the importance of data security and user data ownership, the importance of decentralized storage will become more prominent. The differences between decentralized storage and centralized storage in multiple dimensions such as storage space sources, bandwidth sources, security, and usage methods have resulted in the current scale and performance of decentralized storage being far inferior to centralized storage systems. Although projects such as IPFS and Arweave have made up for their different shortcomings through different “decentralized” design methods, and introduced token incentives to expand the total network storage scale, the interests of the project parties and users are inconsistent with factors such as the cost structure. The rapid expansion of the project is hindered, and its ecological construction is still in the early stage. In the future, it is necessary to find a balance between cost, number of nodes, and number of users to achieve a balance between scale and performance.
Comparison of decentralized storage solutions
We expect that with the opening of the Meta Universe and the Web 3.0 era, the “flywheel effect” of the network will enable decentralized storage to usher in a wave of application explosions. We look forward to the improvement of storage technology and the optimization of economic incentive models. The track can have a long and stable development.
- Cloud Storage Market (2021), Sneha Korad Rachita Rake, Vineet Kumar, https://www.alliedmarketresearch.com/cloud-storage-market
- Distributed storage-a blue ocean war in the hundreds of billions of storage market, https://www.8btc.com/article/613165
- 2020 Distributed Storage Industry Chain Research Report (2020), Jiao Shike https://www.jinse.com/news/blockchain/835473.html
- A ‘more revolutionary’ Web(2006), Victoria Shannon，https://www.nytimes.com/2006/05/23/technology/23iht-web.html
- What Is Web 3.0 & Why It Matters （2020）， Fabric Venture，https://medium.com/fabric-ventures/what-is-web-3-0-why-it-matters-934eb07f3d2b
- IPFS and Friends: A Qualitative Comparison of Next Generation Peer-to-Peer Data Networks (2021), Erik Daniel and Florian Tshorsch
Posted by:CoinYuppie，Reprinted with attribution to:https://coinyuppie.com/decentralized-storage-the-intersection-of-metaverse-and-web-3-0/
Coinyuppie is an open information publishing platform, all information provided is not related to the views and positions of coinyuppie, and does not constitute any investment and financial advice. Users are expected to carefully screen and prevent risks.