Compound Independent chains make the most of Substrate Infrastructure provided , The integration of on chain code and off chain code and the interaction with other chains are realized .

Original title :《 Why? Compound choice Substrate Develop independent chains 》
writing :Mike Tang
source :Web3 Explorer

Compound stay 2021 year 3 In June, it officially launched its independent chain network Gateway ( Original name Compound Chain).Gateway It's a separate chain , Its goal is to become a unified Defi hinge , Achieve cross chain Defi—— You can pledge the chain A Assets on the Internet , In the chain B Borrow money —— So as to get through all the assets of the chain , Improve the utilization and liquidity of network assets .

Gateway It is based on the following judgment : The future world must be a multi chain world , That is, no blockchain can solve all the problems . There will be many chains in the future , There are many possibilities . So people's assets will be distributed to various independent platforms . How to improve the utilization of these scattered assets , Namely Gateway Problems to be studied and solved .

Gateway System architecture

Compound Why do you choose Substrate Develop independent chain ? Detailed explanation Substrate Development mode Gateway Architecture diagram

As shown in the frame composition ,Gateway By targeting various chains (Peer Chain) Develop specific Starport (Peer Chain A combination or component of ), To connect all the chains .

This article will not be published in Gateway Of Defi Too much narration on related things , This paper is mainly about Gateway Why do you choose Substrate Make a brief analysis of the development .

Why choose Substate

Compound Official Statement

We chose Substrate so that we could focus on building application code, instead of inventing consensus algorithms; it』s a modern framework built on a modern language, Rust.

We choose Substrate So that we can focus on building application code , Instead of inventing consensus algorithms ; And it's a modern framework , Built on Modern Language Rust above .

Rust Language is a modern security oriented programming language . About Rust Good word of mouth for language , This article will not repeat , Interested friends can read 《2020 Developer survey report :Rust Become the most popular language again 》.Rust It has almost become the preferred language for blockchain development today , such as :Polkadot, Near, Solana, Dfinity,FileCoin The bottom and so on are Rust Realized .Rust Language has the following remarkable characteristics :

  • Memory safety : Its unique ownership and life cycle design , In theory, there is no memory error . Solve the underlying system defects in 70% The problem of
  • Concurrent security : In concurrent programming ( Multithreading , More cooperation ) in , It can guarantee the security of concurrency
  • High performance : And C/C++ It's the same level of performance , At the top of the mainstream language
  • Zero overhead abstraction : This feature makes Rust Can connect seamlessly C The existing ecology of language , And no performance loss
  • Strong abstract expression ability :Rust It draws on the expressive power of many functional languages , Make it more expressive
  • Modern engineering facilities :Cargo and This modern auxiliary system , Give Way Rust It's easy to build complex projects
  • An active community :Rust From the beginning, it was developed iteratively in community mode , This is actually very interesting , And go This kind of centralized development is significantly different

understand Rust After these characteristics of , Let's understand 「Rust Language has almost become the preferred language for blockchain development .」 This sentence seems very natural . Security is the first priority of blockchain itself , With this Rust It's the same design concept . A lot of computation has to be done in the blockchain , Also need high performance , this Rust It's also our duty . Blockchain systems are generally complex , There is a high demand for engineering related infrastructure , and Rust Powerful engineering design , Let it be in the team 、 Open source community development , Especially suitable for large-scale project collaboration .

Known for its emphasis on safety Paritytech company , use Rust Language developed Ethereum's parity client . Then continue to use Rust Developed new projects Polkadot, Developing Polkadot In the process of , A major decision has gradually taken shape : All the functions of blockchain , Break it down into abstract design , To an open source 、 In a common blockchain framework , And use this framework as a tool , structure Polkadot product . And the major achievements of this blockchain framework , It's the protagonist of this article ——Substrate.

Substrate It's a use. Rust Language development for the purpose of universal blockchain development framework . Its design elements , Like cryptography algorithms 、 Storage structure MPT Trees , Account system, etc , Most draw on the most successful Ethereum infrastructure ever ,( This is understandable ,Paritytech I started as an Ethereum client ,Gavin Wood And one of the co founders of Ethereum ). A framework , To be universal , It needs to be highly abstract . And the cost of being highly abstract tends to be structurally complex , Not easy to use . therefore Substrate It also provides a lot of DSL ( Domain specific language ), Convenient for beginners to learn to use . Briefly sum up ,Substrate It has the following characteristics :

General purpose oriented . It's designed for general purpose , Instead of developing for a certain chain SDK. Every team can use Substrate Develop a completely independent chain independent of any existing network ( such as , Use Substrate The developed blockchain can work with Polkadot irrelevant , This is also Paritytech One of the design goals of ) Comprehensive function . It can cover almost all scenarios of blockchain , It can be said that it is the most comprehensive blockchain framework on the market Runtime Code compiled into wasm perform .Wasm It's the mainstream in today's blockchain industry VM Bytecode selection Super customizable .Substrate It's a bunch of scattered components , Under a set of specifications , Free to replace components , Free combination .

Engineering design is about making choices , When it emphasizes one aspect , per contra , There will be some compromise .Substrate It also has its own shortcomings :

  • Some new concepts are introduced ( such as :extrinsic), Need to relearn to understand , There is a certain threshold to start with
  • Emphasis on universal , Leading to over abstraction in some ways , Code vision is more complex ( For example, there are many generic parameters ). And these abstractions will Rust The high-level grammatical features of , The code is noisy
  • The whole project has a large number of codes , There are a lot of packages to rely on ( There are 1000 Multiple ), It takes a long time to compile ( Ordinary notebook more than dozens of minutes ), High performance requirements for development machine

As you can see above , The main problems are Substrate The design goal of —— Universal —— The objective dependence brought by itself is caused by complexity .

One flaw cannot obscure the splendor of the jade , As one of the strongest blockchain development frameworks so far ,Substrate It is welcomed by more and more innovative teams ,Compound Team choice Substrate Conduct Gateway It's only natural that we should develop our products .

Substrate Functional components of

Compound Why do you choose Substrate Develop independent chain ? Detailed explanation Substrate Development mode Substrate Overall frame diagram

As you can see from the architecture diagram ,Substrate There are several major components :

  • P2p Networking P2p The Internet
  • Runtime Runtime
  • Storage Storage
  • Consensus Consensus
  • RPC Remote procedure call
  • Telemetry Client monitoring tools

P2p The Internet , It is a necessary subsystem of blockchain system . Multiple nodes pass through p2p Connect to form the same network . Sending messages between nodes to deliver data , Message propagation in the network may arrive through multiple relay nodes . stay Substrate in , It uses rust-libp2p (, This project is also mainly composed of Paritytech In charge of maintenance .

Runtime, Runtime , It is the implementation part of the business logic of the blockchain . That is to say, what practical things to do with blockchain , It needs to be written here .Substrate Support will Runtime Code compiled into wasm Bytecode or native code Two modes of operation ( With wasm vm The increase of running speed , future native code Patterns can be abandoned , It also reduces some of the internal complexity ). stay Runtime It can be operated in Storage, Change of implementation state , therefore Runtime The whole is also called the state transition function (State Transition Function, STF).

Storage, Storage subsystem , It is an indispensable part of the blockchain system . stay Substrate in ,Storage For persistence Runtime The logic of change to state , At the same time, we also support the foreign RPC State read interface ,Runtime Medium Event Before sending out , Will also be in Storage For a short time . The goal of the consensus system , It's also true Storage The state of being in agreement .Storage The subsystem at the bottom uses kv database rocksdb or paritydb.

Consensus, Consensus subsystem , Used to agree on the state of the blockchain among the participants in the network , That is to say 「 Consensus 」. Because it's a distributed system , The state of the so-called blockchain , There is no macro perspective of God to see a unified macro state . This state , It's actually the state of each node , Each node has its own local state view , There is also a 「 Local 「 Global view of . Each node compares the local view with the global view , Make a decision . Consensus system should reach a consensus state on each node , To push the system forward .

RPC, Remote procedure call , The interface used to provide access outside the node . One Substrate node It can exist as a service , The outside world can go through these rpc Interface access node Local status information or to node Submit a change request .Substrate At the same time provide HTTP and Websocket Two kinds of rpc passageway .

Telemetry, Client monitoring tools , To collect node Operation information of , Sent to the remote Prometheus The server .

Substrate Development model

Substrate It's a common development framework . It provides three development styles for developers at different levels .

Use it directly Substrate Self contained node Chain up

For those who want to start the chain quickly , Developers of experience effects , You can use it directly Substrate Preset node The implementation of the . Just modify one JSON The configuration file , You can run , The following contents can be customized :

  • Status information of the founding block
  • Account Account
  • balance Balance
  • Staking Proportion, etc

Runtime FRAME pallets Development of

This development model is developed by writing Runtime Medium pallet Code , Implement business logic to pallet in . Then I will customize pallets And what it depends on Substrate Self contained pallets Compile them together into wasm Bytecode runs , This is most of Substrate Developer's choice .

be based on Substrate Core The depth of the custom

Substrate Has been implemented as a decentralized component , Do a full abstraction and decoupling . For some advanced developers , In some specific scenarios , These components can be completely reassembled from the bottom , Achieve deep node The custom of . such as , It can be done :

  • Using different cryptographic curves and hashing algorithms
  • Using different serialization methods
  • Replace different consensus algorithms
  • To remove completely FRAME Layer code , Write business in another language , As long as it's guaranteed to compile to wasm And follow Substrate The standard of
  • wait

The figure below shows the relationship between development difficulty and technical flexibility at three levels .

Compound Why do you choose Substrate Develop independent chain ? Detailed explanation Substrate Development mode

Use it directly Substrate Node The most simple , But the most inflexible . be based on Substrate Core Development is the most flexible , But the hardest . Conduct FRAME pallets Development is in the middle . Most of them Substrate The way developers should be .

Substrate The excellence of

Substrate There's a lot of excellence in the design of , Let's get to know .

Upgradeable without forks Runtime

because Substrate Of Runtime Code compiled as wasm function , then wasm Bytecode itself is directly submitted to the chain as transaction data , And then through the chain itself p2p It's spread all over the Internet , Update the business logic . Each node receives an updated version of wasm After bytecode , Update it to the code snippet , Use a new version of... After a block wasm To execute logic .

With this hot update code mechanism , The upgrade of business code will no longer cause bifurcation ( Soft bifurcation or hard bifurcation ) 了 , in other words , It won't be because of technical objectivity , Leading to the bifurcation of the network ( It's OK to make artificial active bifurcation ).

It should be noted that , This upgrade is limited to wasm What bytecode can cover —— That is to say Runtime The code in —— The upgrade . If it changes node The code itself ( namely Runtime The outer part ), All nodes still need to be informed to do it manually or devops Replace .

Replaceable cryptography Library

Substrate Several cryptographic curves are supported at the same time :

  • Ed25519
  • SR25519

At the same time support several Hash:

  • Blake2
  • xxHash

Developers can choose to use it according to their own needs . If you don't have what you want , Add new curves and Hash Functions are not difficult either .

Rich in layers Account System

Substrate Of Account Key There are three levels :

  • Stash Key
  • Controller Key
  • Session Keys

Stash Key It's a deposit account , The private key part should be stored in the cold wallet as safely as possible .Controller Key Used to control Validator Parameters of , It's also Stash Key An intermediary agent account of , Very useful when updating the verifier collection .Session Keys Used to sign consensus related messages .Session Keys There can be multiple , Each has its own special purpose , It can be used in combination , Such as :

impl_opaque_keys! {

pub struct SessionKeys {

pub grandpa: Grandpa,

pub babe: Babe,

pub im_online: ImOnline,

pub authority_discovery: AuthorityDiscovery,



The above code will 4 Independent Session Keys: Grandpa session key, Babe session key, ImOnline session key, AuthorityDiscovery session key Put it together to make a big SessionKeys.

Can be Session Keys Comprehend Validator Unique identification during operation (Identification).Session Keys Every time I pass by Session It's better to replace it , This ensures maximum security .

Substrate Through this layered Account Key The design of the , It not only ensures the safety , It also provides full flexibility , It can basically cover the requirements of all application scenarios .

Abstract switchable consensus engine

Substrate Designed a consensus framework , This consensus framework is very good . It will be the production of blocks (proposal) With the final (finalize) Separate , It also contains Nakamoto class ( It's just a matter of probability , There's no certainty ) Consensus and BFT class ( With certainty ) Consensus .

Substrate The following algorithms are provided for block output :

  • Aura:slot Pattern , In a known authority set in , Use round robin The pattern turns out to be a block
  • Babe:slot Pattern , In a known authority set in , Using verifiable random functions VRF Randomly select the block node ( Every slot There may be more than one )
  • Pow: The workload proves that

These block algorithms , without Finalize Gadget coordination , It's just a matter of probability (finalization), And there's no certainty .Substrate Provides Grandpa This Finalize Gadget, Used to determine .

The block algorithm and the knock-out algorithm can be used freely . So there is Aura/Grandpa, Babe/Grandpa, even to the extent that Pow/Grandpa These combinations . And in situations where there's no need for certainty , Of course, you can not use Grandpa.Substrate Gives developers full freedom .

Substrate The consensus framework also provides some other infrastructure :

  • Fork Choince Rules: Longest Chain Rule or GHOST Rule, Used to determine if the chain has bifurcations , How to choose the best chain algorithm
  • Import Queue: Import queue
  • Block Import Trait: Block import interface
  • Block Import Pipeline: Block import pipeline

Substrate Also provided in Runtime The function of coordinating and controlling the consensus process . such as , stay Pow It is difficult to adjust during the operation , stay PoA To decide whether the next round will go to , stay PoS Dynamic modification in Stake And so on .

stay Substrate On top of this complete set of infrastructure , If his own consensus engine can't meet the needs of developers , Developers can also develop their own consensus engine according to their specifications , Introduced into the framework to use , And in Runtime Appropriate control of .

Off-Chain characteristic

Off-Chain Characteristic is Substrate It's a pretty strong infrastructure that's available in . After all, for blockchain , The logical operation space on the chain is very limited , Some things have to be done through the chain . In the absence of Off-Chain Worker (OCW) Before , This kind of thing , It's usually done by the oracle Oracle To complete . Oracle is an external service , Through blockchain nodes RPC The interface submits the transaction to the blockchain so as to transmit the external information to the chain . This way, though feasible , But it's in security 、 Integration 、 Scalability and infrastructure efficiency , Still not good enough .

In order to make the data integration under the chain more secure and effective ,Substrate Provides off-chain Related features . The architecture is shown below :

Compound Why do you choose Substrate Develop independent chain ? Detailed explanation Substrate Development mode Off-chain Worker Architecture diagram

Off-chain Features consist of three components :

  • Off-Chain Worker
  • Off-Chain Storage
  • Off-Chain Indexing

Off-Chain Worker It is used to realize the logic under the chain . Its code is similar to Runtime The code is written together , And compiled into the same wasm In a bytecode string , Spread all over the Internet in the same transaction . But in execution ,Off-Chain Worker The code for is independent VM Implemented in , I.e Runtime The execution of logic is completely isolated . say concretely ,Off-Chain Worker It can realize the following functions :

  • Submit the result of calculation to the chain in the form of transaction
  • Contains a full-featured HTTP client , Access to data from external services
  • Access to local node Of keystore, This allows you to verify and sign transactions
  • Access to local KV database , And at all off-chain worker Share this database in
  • Local, secure entropy source , Used to generate random numbers
  • You can access the local time of this node
  • Sure sleep and resume Work

Off-Chain Storage It's a logically independent storage space under the chain , With... On the chain Storage It's completely isolated . It has the following characteristics :

  • Can be Off-Chain Worker Read and write
  • Stored in node Local , It won't be delivered to other nodes in the network , Will not participate in network consensus
  • By all running at the same time Off-Chain Workers share access ( So lock operation is needed ). therefore , It can be used in different Workers Communication between
  • Can be Runtime Code writing , But I can't read . therefore , Based on it, we can realize some interactive functions on and off the chain
  • Can be wasm Outside the environment node Code reading in , So it can be RPC Read

Off-Chain Indexing Provided in Runtime Environment , towards Off-Chain Storage The ability to write data . But can't read Off-Chain Storage Data in . This makes it possible for some new programming paradigms .

There are others , such as , Perfect OCW Integration testing framework and so on .

Substrate Of Off-chain The features are very powerful , Impressive .

Complete and flexible Gas It's a computer system

Friends who have experience in Ethereum development know that ,Gas The fee mechanism is a very successful design , It is very important for the safety and stable operation of the chain . Almost all subsequent blockchains have directly learned from this design . And in the Substrate in , Provides a very complete and detailed mechanism and configuration parameters to help developers design their own Gas Cost algorithm .Substrate The following is built in Gas Calculate and configure parameters :

  • Includsion Fee: contain length_fee and weight_fee
  • Fee Multiplier
  • Additional Fees, contain Bonds、Deposits、Burns And Limits
  • Default Weight Annotations
  • Dynamic Weights
  • Post Dispatch Weight Correction
  • Custom Fees
  • Custom Inclusion Fee

We won't explain each item in detail here , Please refer to this Website .

You can see ,Substrate Yes Gas The design of fee calculation is very comprehensive , It's even a little complicated . The goal is still to achieve the goal of a common blockchain framework —— There should be all of them , And it needs to be out of the box .

Substrate Very suitable for starting independent chains or for Web3.0 Applied Appchain, Generally, these chains will work hard on the user experience . For example, for ordinary users of the application , In the process of using the service , You may not realize Gas The existence of fee , In this case ,Substrate The above abundant Gas Fee mechanism , Can do Gas Set the fee to zero or set it to pay on behalf . This kind of feature has a chance to make Substrate Become the most suitable for Web3.0 App One of the development frameworks of .

Runtime API And RPC Integrate

Substrate A set of RPC Extension framework , So that developers can ( stay Substrate In addition to the interface provided by default ) Expand and develop your own RPC Interface . because RPC The implementation code is in node in ,Runtime outside , So in theory , Can be in Substrate Develop full functionality in ( Do anything ) Of RPC service . This makes Substrate Become a powerful RPC Development framework .

And often we need to communicate with Runtime Interact with the state of , It needs to be used Runtime API 了 .Runtime API yes Runtime The bridge between the inside and the outside , It can also be said that it is a bridge between the chain and the chain .

therefore RPC And Runtime API combined , Then you can get through all the processes from external requests to getting the status on the chain . With custom Runtime API Function and RPC Expand interface functions , Gives developers great flexibility and possibilities . Also for the Substrate Become an integrated Web3.0 Lay the foundation for the development framework .

Gateway How to use Substrate

To find out Gateway How to use Substrate Of , We pull it directly Source code Let's make a brief analysis .

You can see , There are the following main contents ( We added a brief note ).

  • ethereum/ In Ethereum Starport Contract code
  • ethereum-client/ Ethereum related type definitions and basic tool functions
  • gateway-crypto/ Gateway Password related basic tool functions used
  • integration/ Integration test code
  • node/ Substrate The custom of node Code
  • our-std/ Gateway Of std Code , It's right sp_std Simple encapsulation , And added a little extra
  • pallets/ Gateway The main business logic code of independent chain
  • runtime/ Substrate Run time build code for
  • types-derive/ Gateway The type related process macro used

You can see , at present Gateway Only to ethereum Of startport Connect , The docking of other chains is still under development , Will gradually join .

The main business code is in pallets/ Under the table of contents , There are three subdirectories under this directory .

  • cash/ The main business logic
  • oracle/ Get the price code from the price feeder
  • runtime-interfaces/ Some runtime interfaces

We from pallets/cash/src/ Look up the . It's a standard pallet file , Its structure has

  • trait Config The definition of
  • decl_storage! Define the storage part on the chain
  • decl_event! Define the event output section
  • decl_module! Define the module implementation part

In the module implementation part , Yes fn offchain_worker(block_number: T::BlockNumber) {} This function . Its implementation is mainly these two function calls :

  • internal::events::track_chain_events:: ()
  • internal::notices::process_notices:: (block_number)

We know ,Substrate Of offchain_worker entrance , Is called when each block is imported into the local state database . That is to say fn offchain_worker(block_number: T::BlockNumber) This function is driven by each block , This function is executed once per block . therefore , Every block you add , The above code will deal with events and notices These two pieces of content , And these two pieces of code are internal Defined in module .

Keep tracking pallets/cash/src/internal/ and pallets/cash/src/internal/ After the document , We can sort out the general logic points :

  • Offchain_worker One call per block
  • stay offchain_worker In the code , Will be called to ethereum-client/ The basic types and auxiliary tools of Ethereum , Hair http Request to Ethereum's event server , Batch access to Ethereum's events, And sort them out
  • The result of processing will be in the form of transaction from offchain_worker Submit to the chain
  • Get into Runtime Business logic

pallets/oracle/ The logic flow is similar : stay offchain_worker in , use http Request to get data from the price feeder address , And then submit it to the chain for use .

After understanding the above process , It's easier to understand Compound stay Ethereum Do on Compound Governance Proposal How does it affect Gateway The set of verifier nodes on the chain has been updated . It still throws information out through Ethereum Events , After the event server collects the event , Cache it .Gateway This way Offchain worker For each new block, go to the event server to batch retrieve the latest events . After getting the latest events , Just iterate according to the corresponding logic . and Substrate To update the verifier set in , Need to use Controler account, as well as SetKeys And so on Substrate Features provided .

Limited to space , So far, our analysis , Here is only a brief analysis of Gateway Pallet The logical flow of thick lines , Then on Offchain Worker Part of it focuses on .

You can see ,Gateway make the best of Substrate Infrastructure provided , It is very convenient to realize the business logic and the integration of on chain code and off chain code , And interaction with other chains . The whole code is very clear , It is worth learning from .


This article , We introduced Gateway The overall architecture of ,Substrate Functional module of ,Substrate The characteristics of the framework , as well as Gateway How to make the most of Substrate Provide infrastructure for independent chain development .

Limited to space , A lot of places can only be reached , Some links are attached at the end of the article , For your further reading .

Reference material

Introducing Gateway

Gateway Document

《 Compound’s Gateway: a deep dive into setting up a validator node. 》

《Rust Become the most popular language again 》

Source link