Eth Chinese station 2021-06-19 00:32:29 阅读数:295
author ：Vitalik Buterin
thank Felix Lange, Martin Swende, Marius van der Wijden and Mark Tyneway Feedback and proofreading .
How much scalability can we extend the blockchain to ？ Is it really like Elon Musk That's what it says “ Block time accelerated ten times , The size of the block has increased tenfold and the handling fee has decreased a hundredfold ”, It will not lead to extreme centralization and violate the essential attributes of blockchain ？ If the answer is no , So how far can we reach ？ What happens if you change the formula algorithm ？ what's more , If we introduce something like ZK-SNARK Or the function of the slice ？ In theory, a fragmented blockchain can continuously add fragmentation , So can we really do that ？
The fact proved that , Whether or not a slice is used , There are important and very subtle technical factors that limit the scalability of the blockchain . Many situations have solutions , But even with the solution , There are also limitations . This article will explore many of these issues .
If you just raise the parameters , The problem seems to be solved . But what will we pay for it ？
Imagine two in the morning , You got an emergency call , From the other side of the world to help you run the mine ( Pledge pool ) People who . From about 14 It started minutes ago , Your pool and a few other people are separated from the chain , And the Internet is still there 79% Calculation power . According to your node , Most chain blocks are invalid . There is a balance error ： The block seems to have wrongly 450 Ten thousand extra tokens were assigned to an unknown address .
An hour later , You and the other two participants in the same small mine 、 Some block browsers are in a chat room with the exchange side , See someone post a twitter link , It begins with “ Announce new on chain Sustainable Development Fund ”.
In the morning , The discussion was widely spread on Twitter and a community forum that didn't censor content . But then 450 A large part of the 10000 tokens have been converted into other assets on the chain , And billions of dollars in defi transaction .79％ The consensus node of , And all the major blockchain browsers and light wallet endpoints follow the new chain . Maybe the new developer fund will fund some development , Or maybe all of this is being done by the leading mines 、 The exchange and its cronies . But whatever the outcome , The fund has actually become a fait accompli , Ordinary users can't resist .
Maybe there's another theme movie . Maybe it will be MolochDAO Or other organizations .
Will this happen in your blockchain ？ The elite of your blockchain community , Including the mines 、 Block browsers and managed nodes , Maybe it's well coordinated , They're probably all in the same telegram Channel and wechat group . If they really want to make sudden changes to the rules of the agreement out of interest , Then they may have this ability . The Ethereum blockchain completely solved the consensus failure within ten hours , If it's a blockchain implemented by only one client , And you just need to deploy code changes to dozens of nodes , Then you can coordinate client code changes more quickly . The only reliable way to resist this kind of social collaboration attack is “ Passive defense ”, And this power comes from a decentralized group ： user .
Imagine , If the user runs the verification node of the blockchain ( Whether it's direct verification or other indirect technologies ), And automatically reject blocks that violate protocol rules , Even more than 90% Of the miners or Pledgors supporting these blocks , How the story will develop .
If each user runs an authentication node , Then the attack will soon fail ： Some of the pools and exchanges diverge , And it looks stupid in the whole process . But even if only a few users run the authentication node , And the attacker won't win . contrary , Attacks can lead to chaos , Different users will see different blockchain versions . In the worst case , The ensuing market panic and possible continued chain forking will significantly reduce the attacker's profits . The idea of responding to such a protracted conflict can in itself prevent most attacks .
Hasu On this point of view ：
“ We need to be clear about one thing , The reason why we can resist malicious protocol changes , It's because of the culture of user authentication blockchain , Not because PoW or PoS.”
Suppose your community has 37 Node operators , as well as 80000 A passive listener , Check signatures and block heads , Then the attacker wins . If everyone runs the node , The attacker will fail . We don't know exactly what the threshold for initiating population immunity against cooperative attacks is , But one thing is absolutely clear ： The more good nodes , The fewer malicious nodes , And we need more than a few hundred .
In order to enable as many users as possible to run the whole node , We'll focus on ordinary consumer hardware . Even if you can easily purchase dedicated hardware , This can lower the threshold of some full nodes , But in fact, the improvement of scalability is not as good as we think .
The ability of the whole node to handle a large number of transactions is mainly limited by three aspects ：
Many uses “ Simple ” The wrong idea that technology has greatly expanded the capacity of the blockchain comes from the over optimistic estimation of these figures . We can discuss these three factors in turn ：
Calculate the force
The four main reasons why the limit is so low are as follows ：
Please note that , Until recently, most of them were aimed at “ Why just 5-10%？” The explanation of this point focuses on a different issue ： because PoW The time of the block is uncertain , It takes a long time to verify blocks , It increases the risk of creating multiple blocks at the same time . There are many ways to fix this problem , for example Bitcoin NG, Or use PoS Proof of interest . But these don't solve the other four problems , So they didn't make as much progress in scalability as many people expected .
Parallelism is not a panacea . Usually , Even clients that look like single threaded blockchains have been parallelized ： The signature can be verified by one thread , Execution is done by other threads , And there is a separate thread processing the transaction pool logic in the background . And the closer all threads are used 100%, The more energy consumed by the running nodes , in the light of DoS The lower the safety factor is .
Now , We often hear widely spread statistics about how much bandwidth an Internet connection can provide ：100 Mbps even to the extent that 1 Gbps The numbers are very common . But for the following reasons , There is a big difference between the claimed bandwidth and the expected actual bandwidth ：
When Starkware stay 2019 When an experiment was conducted in , They're trading data gas For the first time since the cost reduction 500 kB Block of , Some nodes can't actually handle blocks of this size . The ability to handle large blocks has improved and will continue to improve . But whatever we do , We still can't get to MB/ The average bandwidth in seconds , Convince ourselves that we can accept 1 Second delay , And have the ability to handle blocks of that size .
As you may have guessed , The main argument here is the same as elsewhere ： The difference between theory and practice . Theoretically , We can buy it on Amazon 8 TB Solid state drive ( It does SSD or NVME;HDD Too slow for blockchain state storage ). actually , The laptop I use to write this blog is 512 GB, If you ask people to buy hardware , A lot of people get lazy ( Or they can't afford 800 The dollar 8 TB SSD) And use centralized services . Even if the blockchain can be installed on a storage device , A lot of activity can also quickly deplete disks and force you to buy new ones .
A group of blockchain protocol researchers surveyed everyone's disk space . I know the sample size is very small , But still ...
Besides , The storage size determines the time required for a new node to go online and start participating in the network . Any data that the existing node must store is the data that the new node must download . This initial synchronization time ( And bandwidth ) It's also the main obstacle for users to be able to run nodes . In writing this blog post , Synchronize a new geth It took me about 15 Hours . If Ethereum usage increases 10 times , So synchronize a new geth Nodes will take at least a week , And it is more likely to restrict the Internet connection of nodes . This is more important during the attack , When the user has not run the node before, the successful response to the attack requires the user to enable a new node .
Besides , There are interaction effects among these three types of costs . Because the database uses tree structure to store and retrieve data inside , So the cost of getting data from a database increases with the logarithm of the database size . in fact , Because the top ( Or previous levels ) It can be cached in RAM in , So the cost of disk access is directly proportional to the size of the database , yes RAM Multiple of the size of the cache data in .
Don't take this picture literally , Different databases work in different ways , Usually the part in memory is just a single ( But it's big ) The layer ( See leveldb Used in LSM Trees ). But the basic principle is the same .
for example , If the cache is 4 GB, And we assume that each layer of the database is larger than the previous one 4 times , So Ethereum's current ~64 GB The state will need to ~2 visit . But if the state size increases 4 Times to ~256 GB, So this will increase to ~3 visit . therefore ,gas The upper limit increases 4 Times can actually be translated into a block verification time increase of about 6 times . The impact could be even greater ： It takes longer to read and write when the hard disk is full than when it is idle .
Now in the Ethereum blockchain , Running a node is already a challenge for many users , Although it is possible to use at least conventional hardware ( I just synced a node on my laptop when I wrote this article ！). therefore , We're going to have a bottleneck . Core developers are most concerned about the storage size . therefore , Great efforts are currently being made to solve the computing and data bottlenecks , Even changes to consensus algorithms , Are unlikely to bring gas limit A big increase in . Even if we solve the biggest problem of Ethereum DoS weaknesses , Also can only be gas limit Improve 20%.
For the problem of storage size , The only solution is stateless and stateful . Stateless makes it possible for node groups to verify without maintaining persistent storage . State overdue deactivates states that have not been accessed recently , The user needs to manually provide proof to update . These two paths have been studied for a long time , And we've started the implementation of the stateless proof of concept . The combination of these two improvements can greatly ease these concerns , And for significant improvement gas limit Open up space . But even after the implementation of stateless and state overdue ,gas limit Or it could only be safely raised about 3 times , Until other restrictions come into play .
Another possible medium-term solution is to use ZK-SNARKs To verify the transaction .ZK-SNARKs It can ensure that ordinary users do not need personal storage state or verification block , Even if they still need to download all the data in the block to defend against data unavailability attacks . in addition , Even if an attacker can't force an invalid block , But if it's too difficult to run a consensus node , There is still a risk of coordinated censorship attacks . therefore ,ZK-SNARKs It can't improve the capability of nodes indefinitely , But it can still be greatly improved ( May be 1-2 An order of magnitude ). Some blockchains are layer1 Explore the form on the Internet , Ethereum goes through layer2 agreement ( Also called ZK rollups) To benefit , for example zksync, Loopring and Starknet.
Fragmentation fundamentally solves the above limitations , Because it decouples the data contained in the blockchain from the data that a single node needs to process and store . Node verification blocks are not downloaded and executed in person , Instead, it uses advanced mathematical and cryptographic techniques to indirectly verify blocks .
therefore , A partitioned block chain can safely have a very high level of throughput that a non partitioned block chain cannot achieve . This really requires a lot of cryptography techniques to effectively replace naive full verification , To reject invalid blocks , But it can be done ： The theory already has a foundation , And a proof of concept based on the draft specification is already underway .
Ethereum plans to adopt Quadratic partition (quadratic sharding), Total scalability is limited by the fact that ： Nodes must be able to handle a single slice and beacon chain at the same time , The beacon chain must perform some fixed management work for each fragment . If the slice is too big , Nodes can no longer handle a single slice , If there are too many pieces , Nodes can no longer handle beacon chains . The product of these two constraints forms the upper bound .
As you can imagine , By cubic partition or even exponential partition , We can go further . In such a design , Data availability sampling is bound to become more complex , But it can be done . But Ethereum doesn't go beyond the quadratic , The reason lies in , From transaction fragmentation to transaction fragmentation, the additional scalability gains can not be realized on the premise that other risk levels are acceptable .
So what are these risks ？
As you can imagine , As long as one user is willing to participate , The chain can be run without division . But this is not the case with the sub block chain ： A single node can't handle the whole chain , So enough nodes are needed to work together on the blockchain . If each node can handle 50 TPS, And the chain can handle 10000 TPS, So the chain needs at least 200 Only nodes can survive . If the chain at any time is less than 200 Nodes , There may be a situation where the nodes are no longer synchronized , Or the node stops detecting invalid blocks , Or there could be a lot of other bad things , It depends on the settings of the node software .
In practice , Because of the need for redundancy ( Including data availability sampling ), The minimum number of safe is more than simple “ chain TPS Divided by nodes TPS” How many times higher , For the example above , Let's set it to bit 1000 Nodes .
If the capacity of the block chain increases 10 times , The minimum number of users will also increase 10 times . Now you may ask ： Why don't we start with a lower capacity , Add when there are many users , Because this is what we really need , When the number of users falls back, the capacity will be reduced ？
Here are a few questions ：
The minimum number of users is 1,000, It's almost no problem . On the other hand , The minimum number of users is 100 ten thousand , It can't be . Even if the minimum number of users is 10,000 It can also be said that it is beginning to become risky . therefore , It seems difficult to justify a chain of blocks with more than a few hundred blocks .
The important attribute of blockchain that users really cherish is Permanence . When the company goes bankrupt or when it is no longer profitable to maintain the ecosystem , The digital assets stored on the server will be in 10 No longer exists within the year . And Ethereum NFT It's permanent .
Yes , To 2372 People will still be able to download and look up your encrypted cat .
But once the capacity of the blockchain is too high , It becomes more difficult to store all this data , Until at some point there's a huge risk , Some historical data will eventually …… No one's storing .
It's easy to quantify this risk . With the data capacity of the blockchain (MB/sec) In units of , multiply ~30 Get the amount of data stored each year (TB). The data capacity of the current fragmentation plan is about 1.3 MB/ second , So it's about 40 TB/ year . If you add 10 times , Then for 400 TB/ year . If we don't just want access to data , And in a convenient way , We also need metadata ( For example, unzip aggregate transactions ), So every year we reach 4 PB, Or ten years later 40 PB.Internet Archive ( Internet Archives ) Use 50 PB. So this can be said to be the upper limit of the safe size of the block chain .
therefore , It seems that in these two dimensions , Ethereum chip design is actually very close to the reasonable maximum safety value . The constant can be increased a little bit , But not too much .
There are two ways to try to expand blockchain ： Basic technical improvement and simple upgrading of parameters . First , Raising the parameters sounds attractive ： If you're doing math on paper , It's easy to convince yourself that a consumer laptop can handle thousands of transactions per second , Unwanted ZK-SNARK、rollups Or slice . Unfortunately , There are many subtle reasons why this approach is fundamentally flawed .
Computers running blockchain nodes are not available 100％ Of CPU To verify the blockchain ; They need a large margin of safety against unexpected accidents DoS attack , They need spare capacity to perform tasks like processing transactions in the memory pool , And users don't want to run the node on the computer and can't use it for any other application at the same time . Bandwidth will also be limited ：10 MB/s Just because you're connected doesn't mean you can process it every second 10 MB Block of ！ Maybe every 12 Seconds to process 1-5 MB The block . Storage is the same , It is not a solution to improve the hardware requirements of running nodes and limit the number of dedicated node operators . For decentralized blockchains , Ordinary users can run nodes and form a culture , Running nodes is a common behavior , This is crucial .
However , Basic technical improvements are feasible . At present , The main bottleneck of Ethereum is the storage size , Statelessness and state overdue can solve this problem , And make it grow at most about 3 times , But no more , Because we want to make it easier to run nodes than it is right now . Blockchain with fragmentation can be further expanded , Because a single node in the block chain does not need to process every transaction . But even a chain of blocks , There are limits to capacity ： As capacity increases , The minimum number of secure users has increased , The cost of archiving blockchain ( And if no one archives the chain , There is a risk of data loss ) It will rise . But we don't have to worry too much ： These restrictions are enough for us to process more than one million transactions per second while ensuring the complete security of the blockchain . But to achieve this without compromising blockchain's most valuable decentralization features , We need to do more .
Link to the original text ：https://vitalik.ca/general/2021/05/23/scaling.html
版权声明：本文为[Eth Chinese station]所创，转载请带上原文链接，感谢。 https://netfreeman.com/2021/06/20210604195029101a.html