Verifiable distribution network: the ultimate solution for blockchain expansion

User 9624935 2022-04-03 05:07:48 阅读数:18

verifiable distribution network ultimate solution

Due to the decentralized nature of blockchain ( That is, no entity controls its operation ), More and more people are looking forward to , Or at least hope , Blockchain plays its subversive potential in more fields . However , Decentralization comes at a price : The blockchain is not modeled by laws and regulations , Unable to handle large or even moderate transactions in a timely manner . for example , Bitcoin handles three transactions per second .

The root of the problem , It is also a limiting factor of blockchain , The reason is that blockchain is based on trust free (trustless) Peer to peer network model . In this model , Information must be disseminated and verified at every hop in the network . without doubt , Cloud distribution network can solve similar performance challenges in other fields ( for example Akamai or YouTube solve web And video transmission performance ), It can also be used to solve the scalability problem of blockchain . The problem lies in , Such a huge centralized infrastructure like cloud disturbs the decentralization of blockchain , Thus eliminating the disruptive potential of blockchain . thus , You can ask such a question : Whether the cloud distribution network can improve its scalability without destroying the decentralization of blockchain ? The answer is yes , The key to the solution is based on existing concepts ( Network neutrality ) Propose a modified advanced version : Verifiable neutral cloud distribution network .

By bitcoin in 2008 Blockchain and cryptocurrency revolution launched in is booming . Although the market value of mainstream cryptocurrencies fluctuates sharply , But there are still hundreds of billions of dollars . A unique feature of blockchain is that there is no centralized governance . Blockchain relies on third-party arbitration ( namely , A global peer-to-peer network consisting of participating nodes that verify and authenticate all transactions ). Purely distributed and decentralized design based on blockchain , Many people think that , This system has disruptive potential in areas other than cryptocurrency , Including medical treatment 、 The government 、 manufacture 、 retail 、 insurance 、 The Internet of things 、 Sharing economy, etc . Many high-tech companies, large and small, are paying close attention to the blockchain field , Analyze how this new technology will affect their existing or future operations .

A major problem with blockchains is scalability . Blockchain system throughput is supported by the system TPS( Transactions per second ) To measure . Bitcoin's current average throughput is 3 individual TPS, and Visa The average throughput of the centralized system is 2000 individual TPS, The daily peak is 4000 individual TPS, The maximum throughput is 5.6 m TPS. No scalability , Cryptocurrency system will be difficult to become mainstream , Blockchain is also unlikely to realize its disruptive potential in any other field .

1. What is a blockchain ?

Blockchain is a public distributed ledger , It stores all past transactions , It is essentially a type of database , It consists of multiple connected nodes in a peer-to-peer network ( Tens of thousands ) Node creation and sharing . In order to reach a consensus on the correct copy of the database , Some rules for writing to the database must be specified . Although the rules may be different , But it generally includes the following :

  • transaction ( Usually a certain amount of cryptocurrency is sent from one user to another ) Must contain a digital signature from the participating node , For authentication .
  • Transactions must be added in order . Transactions will not be added to the ledger alone , contrary , They are added in batches , Called a block . for example , Bitcoin blockchain requires each new block to contain a hash “ problem ” The answer , The answer is uniquely determined by the last transaction block on the chain and the current block being added .
  • Adding blocks to a blockchain is both expensive and competitive . Each participating node that wants to add blocks to the blockchain either invests in cryptocurrency , Or invest money , for example , The hash required for bitcoin “ problem ”. Such participating nodes are called miners , The process of adding new blocks to the blockchain is called mining .
  • The longest blockchain is the latest version . When this rule is combined with the previous rule , This makes the cost of successfully forging blockchain very high . Even if you copy an existing blockchain and try to modify the last few blocks , It's also very expensive . Once the block is sufficiently confirmed on the network , Deleting or modifying blocks becomes mathematically impossible . therefore , Transactions can only be added to blockchain , They will never be deleted .
  • Independent verification . When the node checks the copy of the blockchain database , It should be able to independently verify that all previous rules have been followed . If each user can independently verify the blockchain , Then all users can reach a consensus on the correct blockchain .
  • Adding blocks to the blockchain can reap rewards . Because it is difficult to write blocks to the blockchain , Therefore, not all nodes will participate in this process . Many users create transactions , Then ask to write the transaction to the network , Users usually pay a certain fee as a reward for miners . Besides , As long as the miners win a round of mining , And have the opportunity to add a block in the blockchain , They can distribute the newly produced cryptocurrency to themselves .
  • The blockchain is forked , Solve through the longest chain rule . Reaching a consensus on the blockchain is not immediate , Sometimes blockchains may diverge ( Different copies of the database ). Bifurcation is that the public ledger of the blockchain diverges after a common history , Different versions of the ledger coexist . The node selects the longest blockchain on the network , Can solve the bifurcation problem .

2. The scalability of blockchain

Before explaining blockchain scalability , Let's first look at how it behaves in reality . chart 1 Sum graph 2 It shows the transaction backlog of bitcoin and Ethereum, two mainstream cryptocurrencies . You can see , Thousands of transactions are regularly waiting for blockchain processing . In order to increase the number of people selected by miners “ Upper chain " The possibility of , user ( voluntarily ) Increased transaction fees . therefore , Transaction fees are by no means trivial , During trading congestion , Transaction fees may increase significantly .

To understand where the bottleneck is , Let's first calculate the blockchain throughput . System throughput depends directly on two parameters : Block size B( namely , Each block can contain the number of bytes of the transaction ) And block out interval T( namely , The average time required for the system to dig out a new block ). In bitcoin ,B = 1 MB,T ~ 600 second , It's about 3 individual TPS. thus , The throughput of blockchain can be improved through the following options : increase B, To include more transactions ; Reduce T, Get out of the block faster ; Or both . The problem is , These parameters cannot be changed at will , I'll talk about it in detail later .

obviously , It is the distributed nature of blockchain that causes these problems . You bet , As long as blocks and transactions can spread instantaneously between nodes , You can quickly dig out a large number of blocks , Until a certain CPU And flash array limitations . However , actually , Blockchain nodes —— Thousands or more —— All over the world . therefore , The network is the bottleneck .

Nodes in the blockchain network communicate in a peer-to-peer manner . Unfortunately , This is related to the following high throughput 、 The goal of low latency runs counter to :

  • Information is transmitted from one node to another ; therefore , Spreading information across the network requires multiple hops . Because every node in the network does not trust other nodes , Therefore, the transmitted information must be verified independently at each hop . This usually involves performing encryption operations on each hop , This increases latency and affects throughput .
  • The performance of nodes in blockchain network varies greatly , This means that a single slow node on the critical path will expand the propagation time .
  • Last , Nodes in peer-to-peer networks are randomly formed ; therefore , They are not organized for optimal data dissemination . This means that the data will be transmitted through the suboptimal path in the network .

therefore , Will a 1MB The block propagates to the bitcoin network 90% The average node needs 11.6 second , This is a 2017 year 3 The average propagation time observed in the month . Unfortunately , It's just part of the problem . Both theory and practice show that , Change the block size B Add one more X The factor also increases the time required for block propagation by the same X factor . similarly , Block out interval T Reduce by one factor X It will also produce exactly the same effect . This means that the block propagation time increases proportionally with the increase of these two parameters .

for example , Increasing the block size ten times will also increase the block propagation time ten times , Make them last longer than 100 second . Again , Increase the block size 100 Times will cause the block propagation time to exceed 1000 second . This propagation time exceeds the out of block interval , As a result, each time a new block is excavated, a bifurcation will occur . actually , In this scenario , The bifurcation will not be solved by continuing to dig out subsequent blocks , contrary , The blockchain will be broken down into ” Bifurcation “、” A forked fork “ and ” Bifurcated bifurcated “, Until the node and the miner don't know which fork is “ Correct ” chain —— So the blockchain collapsed . This is the scalability problem of blockchain caused by network bottleneck .

3. Cloud distribution network (Cloud-Delivery Networks)

Cloud distribution network is solving Internet Very successful in terms of performance issues on . Such networks distribute content through a huge infrastructure , This infrastructure can consist of thousands of servers around the world ( for example Akamai). Besides , Cloud distribution networks perform extensive network and server measurements , And use these measurements to redirect customers to nearby servers . This helps the Internet run on a huge scale . for instance , only YouTube Have more than 10 Million users , In North America, the peak hours at night are as high as 70% The network traffic comes from Netflix and YouTube Streaming video and audio websites . If there is no cloud distribution network , It's impossible .

This is in sharp contrast to the current situation of blockchain . actually , As explained earlier , Spread a message through the blockchain network 1 MB The block is a time-consuming task , And increasing the size of the block may lead to unrecoverable problems . However , The cloud distribution network can send... Every second TB Level of data , This is considered normal . Can such a network be used to expand blockchain ?

without doubt , Cloud distribution network can improve the performance of blockchain . The problem is trust . In the blockchain ecosystem , The node does not trust its direct peer , that How can it trust a cloud distribution network that is much more powerful than any single node ? The cloud distribution network is a transaction that can review the blockchain network 、 A centralized system of blocks or miners . for example , Cloud distribution network administrators can according to their own policies 、 Business interests or legal requirements , Reject blocks containing unauthorized traders , Or an unauthorized miner's block .

therefore , Here's the key , Is it possible to make cloud distribution networks trust free (trustless), In this way, they can be used to expand the blockchain network , Without exercising the censorship and other powers mentioned earlier in this article . This concept is called Verifiable network neutrality (provable net neutrality). This paper does not discuss its formal definition in depth , Instead, it outlines the key attributes related to this concept .

First , The network should not censor information based on block content . secondly , The network should not censor nodes . Third , The node should be able to continuously verify the above two properties , And when the network misbehaves , You can abandon and replace the network . How to enable these properties ?

4. A verifiable neutral blockchain distribution network

Consider a cloud distribution network , Its goal is to make the blockchain system ( It's not necessarily just cryptocurrency ) It can be extended to thousands of uplink transactions per second . Besides , Another goal is to provide scalability for many cryptocurrencies and blockchains at the same time , Support the distributed blockchain system in a verifiable and neutral manner using the global infrastructure . That's what's called Blockchain distribution network BDN(blockchain distribution network). This section outlines the trust model of the system , Then it describes the key mechanisms needed to implement the neutral attribute .

4.1 Reverse trust model

Blockchain distribution network (BDN) Our trust model is based on two observations : First , Long time block propagation can never significantly improve trust free (trustless) Blockchain peer-to-peer network ( Such as COINS ) extensibility ; secondly , Small centralized system can be well expanded , That is, trust a small number of participating nodes and hand over the control of blockchain packaging transactions to them ( for example Ripple and EOS).

However , This centralization destroys one of the most significant aspects of the blockchain : To deal with ( Or deal packaging rights ) Distributed and decentralized control . Give the right to package blockchain transactions to a limited number of participating nodes , This allows participating nodes to participate in user 、 Collusion between nodes and miners 、 Review and differentiate (discrimination). The limited number of participating nodes also reduces the number of nodes that malicious nodes have to pay to control the system .

4.2 Verifiable network neutrality

In short , Blockchain distribution network (BDN) All blocks can only be fairly propagated to all blockchain nodes , And due to the audit of blockchain nodes ,BDN Cannot treat blocks differently (discrimination), Still connected point-to-point .

4.3 Encrypted block

To prevent blockchain distribution networks (BDN) Block the propagation of any block according to its content , Blocks are propagated after encryption ( chart 3 Step in 1).BDN The size of the encrypted block has also changed , Hides the number of transactions and their total size . After the block is propagated , The receiver notifies the sender by sending the hash of the block ( chart 3 Step in 2). Last , Publish the encryption key of a block , And spread directly on the blockchain peer-to-peer network ( chart 3 Step in 3). The encryption key is very small , Just a few bytes , Allow it to spread directly and quickly over peer-to-peer networks , And BDN Can't stop it .

4.4 Indirect relay

To ensure blockchain distribution network (BDN) Individual nodes are not prevented from propagating their blocks , Nodes can not propagate blocks directly to BDN. For a person who has not been BDN Block of propagation ( chart 4 Step in 1), The sending node will send it ( block ) Propagate to a peer node on a peer-to-peer network ( chart 4 Step in 2), This peer node will put it ( block ) Forward to BDN( chart 4 Step in 3), Yes BDN Confuse the origin of the block . for example , A node digging up a block in China can forward it to a node in Europe , Then the node passes through BDN Send the block . In addition to indirectly relaying blocks to BDN outside , Nodes can also request that their peer nodes will come from BDN The incoming blocks are relayed to them . This ensures BDN Nodes cannot be treated differently by delaying the distribution of blocks , Because nodes do not need to communicate with... In order to benefit from their services BDN Direct interaction .

4.5 Audit through test blocks

Although blockchain distribution networks (BDN) I don't know which node dug the block , But it may try to stop or delay blocks from certain node subsets , This affects all blocks they relay . To detect and prevent this behavior , The node must be able to continuously monitor BDN Service for . This monitoring is done by allowing nodes to encrypt invalid blocks 、 The test block is sent directly to BDN( chart 5) And measure the time required for the peer node to report the arrival of the test block .BDN It is not possible to use discriminatory policies only for valid blocks , And faithfully spread the test block , Because the two test blocks cannot be distinguished before the key is released .

therefore , By using traffic encryption and indirect traffic relay , And explicit auditing BDN, Blockchain nodes can limit BDN Misbehavior , Effectively will BDN The authority of the administrator is the same as BDN Infrastructure decoupling . If BDN Stop distributing blocks completely , Or distribute blocks to only a small number of nodes , Blockchain nodes can be abandoned BDN.

Because nodes often use test blocks to infer the best source of receiving blocks , By BDN Discriminating nodes will simply receive blocks from their peers . therefore , If BDN Malicious discrimination against many or all peer nodes , Peer nodes will simply form their own peer-to-peer networks , Until a different system takes over . Besides , If discrimination is caused by large-scale system failure , Once the fault is solved , The peer node will return to use BDN.

5. performance

Essentially , Blockchain distribution network (BDN) Deployed broadcast primitives , This means that it can effectively transfer data from one source node to all other nodes in the blockchain network . Contrary to peer-to-peer networks ( In Peer-to-Peer Networks , Each blockchain node is connected to many other nodes , These nodes are usually distributed all over the world ), Blockchain nodes replace this one-to-many communication with one-to-one communication . This is because blockchain nodes are connected to a single node BDN The server .

For the larger TPS rate , Using a single connection is more conducive to scalability than using multiple connections . In order to audit effectively BDN, Blockchain nodes must be connected in a peer-to-peer network . However , Most of the data is in BDN Back and forth between . Here are BDN Several ways to help expand blockchain .

5.1 Transaction caching

In the blockchain system , Like bitcoin or Ethereum , Each node receives two transactions : The first is the original transaction , Initially spread through the Internet , The second time is to write the transaction into the block .BDN Can effectively distribute transactions through the cloud , And index them , Then use the index when transmitting the block ( Not the original deal ). This effectively compresses the block size 100 Many times , Suppose the original transaction is about 500 Byte length , And the index can be 4 Bytes or less .

Transaction caching is an idea that already exists in the blockchain ecosystem , It has been adopted by some projects , But it is only deployed by endpoints , Instead of network deployment . therefore , Considering that not all transactions in the pure blockchain system reach all end nodes , Even a slight asynchrony can lead to a significant increase in block size ( Not all transactions are ” Compression of the “); therefore , Performance can be affected . contrary ,BDN Effectively transmit and index blockchain transactions .

5.2 Direct routing

Different from blockchain nodes ,BDN Cannot check the validity of blocks flowing through the network , Because blocks are encrypted . This facilitates the rapid transmission of data blocks over the network . especially , In a BDN Before the node receives all bits of a block ,BDN It is already possible to start transmitting the bits of the received block to other parts of the network . This is called direct routing , It has been widely used in network switches for decades . For block propagation , It can still significantly speed up data transmission , Especially when the data block is large . 5.3 transaction Incast problem

The transaction needs to be broadcast in the blockchain network . In the absence of BDN Under the circumstances , When TPS When the speed is high , There will be a so-called transaction incast problem : Receive the same transaction from multiple sources at a higher rate . This will significantly affect the resources of the node , Affect the performance of the whole blockchain .BDN Eliminate the problem , Because most of the data ( Including transactions ) It's all in a single BDN Propagation between servers .

Incast is a many-to-one communication pattern commonly found in cloud data centers.

6. Research on the scalability of blockchain

Other methods to improve the scalability of blockchain are described below .

6.1 Off-chain Solution

Another way to improve scalability is to use off-chain transaction , for example , Lightning network , Its purpose is to reduce redundant data on the main chain . Generally speaking , One off-chain The solution will open a payment channel between the two sides of the transaction , That is, let the buyer and the seller exchange funds , Record the intermediate balance at the same time , Then the transaction settlement is carried out on the blockchain .

BDN The proposal is agnostic to such a solution (agnostic). As a off-chain Expand the solution , In essence, it still needs the up chain function . Besides , The potential expansion benefits are multiplied . If the underlying blockchain can support more than before 1000 Times the number of transactions , also off-chain Transactions will increase throughput 1000 times , that , The throughput of blockchain has doubled 6 An order of magnitude . 6.2 On-chain Solution

On-chain Solutions often involve modifying consensus agreements in some way , To achieve higher throughput . One way is , That is, slicing technology (sharding), Divide the blockchain into several smaller pieces , A whole node only needs to track one fragment , Instead of a complete blockchain . These pieces are interlaced , Careful maintenance , In order to preserve the original security attributes of the blockchain . There are many other ideas in this field . Although these methods show some potential , But their robustness 、 Security and usability remain to be seen in practice .

For all that , In a faster network layer , be-all on-chain Solutions will perform better , This is also BDN Where to improve its performance . in fact , In each distributed consensus protocol , Each node that follows the agreement must reach the same decision . therefore , Each such peer must be independent of the consensus protocol , Get information about each transaction in the system .BDN Focus on this particular problem , This problem is essentially a broadcasting problem , Because every valid piece of information must be propagated to every peer node in the system . therefore ,BDN Unknowable to local consensus agreements (agnostic), It can significantly improve the performance of any blockchain . 7 summary

Verifiable neutral cloud is undoubtedly a feasible solution to improve the scalability of blockchain . By optimizing the transport layer , It can not only fundamentally improve the throughput , And it can significantly reduce the delay . in fact , The delay distribution in today's data center has been biased towards most microseconds , Milliseconds exist only at the tail of the distribution .BDN There is no reason why inbound outlets do not achieve similar performance .

In these BDN Add dedicated optical fiber infrastructure between access points , Will further reduce throughput and latency , So as to establish advanced BDN Backbone network . However , The key to achieving this vision is to build trust in the underlying network infrastructure through the blockchain ecosystem . This is achieved through a verifiable neutral network design that decouples administrative authority from infrastructure .

Translation notes : Capacity expansion 、 Extensibility 、 Scalability 、 Performance and other words are different expressions of the same problem in the context of blockchain .

The author of this article :

Aleksandar Kuzmanovic He is a professor of computer science at Northwestern University . His recent research includes content distribution networks 、 Network neutrality and blockchain . He's a startup bloXroute Labs Co-founder of , And served as the chief architect of the company .

Silence time

Silence is the language of God,

all else is poor translation.

Silence is the language of God ,

Everything else is a poor interpretation .

—— rumi (Rumi)

版权声明:本文为[User 9624935]所创,转载请带上原文链接,感谢。