Using the graph to realize the 10000 times speed optimization of DAPP

Extremely slow front end experience

stay How to issue a NFT( Next ) The official website of is under construction , We found that NFT List page , The display speed is very slow , Always have to wait 5、6s, This experience is unacceptable , Let's first analyze why this page is so time-consuming , The following is the process of obtaining data :

As shown in the figure , We need to keep circulating requests to get tokenId Corresponding tokenURI, And then through tokenURI The request for metadata To parse the data , Every cycle is 2 Time http Communication is based on low delay http Communication takes time 300ms, Each cycle will consume 600ms, And this number will increase with NFT The quantity increases linearly , Our page takes time 5、6s It's because of us NFT only mint 了 10 about , Normal Online NFT Usually there will be close to 1 m toknId, Then the overall time consumption will reach 100 minute , No user can tolerate such a speed . So we must need other solutions to optimize this process .

Our ideal process is :

Ideally, the front page only needs 1 One request to obtain the required data , Instead of making multiple requests . This requires 1 The back-end services under the chain help us do similar logic and store data , When the front-end page needs to be displayed, request the data stored in the back-end service , and TheGraph It's such a service , It is different from the back-end service we built ourselves ,TheGraph It's decentralized , This greatly avoids the centralization risk of our single node .

TheGraph Introduction to working principle

TheGraph During initialization, we will scan the events we need from all historical blocks , Every scan to 1 One will call the event handler we wrote , Until you reach the latest block , After reaching the latest block TheGraph Will listen to each new block and find out if there are events needed . We usually in the event handler function , Perform logical operations and finally store . At this point, the data we have processed will be saved in TheGraph in . When the front-end needs this data, you can use GraphQL Request to get . The complete process is shown in the figure below ：

Ethereum event event

Before that Ethereum Technology Series - Ethereum data structure It has been introduced in that Ethereum has 3 tree , State tree , The trading tree , The receipt tree . Ethereum events are stored in the receipt tree . Write... In the smart contract 1 The way to create an event is to use event Keyword definition , Use emit Keyword send .

// Defining events event Transfer(address indexed from, address indexed to, uint256 indexed tokenId);// Send events emit Transfer(owner, address(0), tokenId);

GraphQL brief introduction

GraphQL It mainly provides us with a more fine-grained way to interact with the server , Compared with the previous REST API, There are several advantages :

1. Reduce field redundancy , Return only the fields needed in the query .
2. Avoid multiple requests for data , Such as the first request for id The second request passed id Scenes with other attributes can be returned at one time .
3. It is more conducive to the separation of front and rear ends , The front end only needs to add fields in the request to obtain the new fields , No back-end coordination is required .

1 individual GraphQL request - Respond to the request as follows :

{ metaDatas(first:1) { id name image owner }}

Respond to :

{ "data": { "metaDatas": [ { "id": "0", "name": "nft-web3-explorer", "image": "ipfs://xxx/0.png", "owner": "0xxxx" } ] }}

Objective combing

Next, let's introduce how we can pass TheGraph To optimize NFT List display . The details are as follows 3 Step :

1. Identify the events that need to be used in the contract , because TheGraph It is triggered based on events in Ethereum , So we need to send events in the right place in the contract .
2. Complete the back-end code -TheGraph Handle and store events , We need to define the storage structure of data , And write a processing function to handle the event when it is received , Store the data according to the defined structure .
3. Complete the front-end code , The request for TheGraph The data of , We go through GraphQL Get the required data from the storage .

Add events to the contract

Since the trigger source is from Ethereum event, Then we definitely need to send... At the right time event, Because we use ERC721 The implementation of the , View the code in mint It will be sent when Transfer event,event With medium mint Address and tokenId, In line with our needs , So we chose to use this event As our trigger .

function _mint(address to, uint256 tokenId) internal virtual { require(to != address(0), "ERC721: mint to the zero address"); require(!_exists(tokenId), "ERC721: token already minted"); _beforeTokenTransfer(address(0), to, tokenId); _balances[to] += 1; _owners[tokenId] = to; emit Transfer(address(0), to, tokenId); _afterTokenTransfer(address(0), to, tokenId); }

TheGraph Code writing ( back-end logic )

Create a project

To TheGrapha Apply for subGraph, Connection required github.https://thegraph.com/hosted-service

Local project initialization

// install graph The scaffold npm install -g @graphprotocol/graph-cli// Initialization project , Select the contract address ,abi file ( Will automatically read from the contract address , Failed to read. It can be uploaded locally ), The Internet ( We still use it rinkeby Test network )graph init <GITHUB_USERNAME>/<SUBGRAPH_NAME> <DIRECTORY>

Define the storage structure

stay schema.graphql Define our storage structure

// Record each time Transfer event ( It's not necessary )type QLTransfer @entity { id: ID! from: Bytes! # address to: Bytes! # address tokenId: BigInt! tokenURI: String!}// Record Metadata data type MetaData @entity { // Each data needs to be unique id, Here we use tokenId id: ID! // hold tokenId The address of owner:Bytes! //metadata In the document name name: String //metadata In the document image image: String}

Generate the corresponding... Through the defined storage structure ts Code to facilitate the direct call of .

graph codegen

Complete the logical processing of event processing

stay mapping.ts Complete our logical processing in , Use AssemblyScript To write (AssemblyScript yes TypeScript Subset ,AssemblyScript standard Reference resources )

export function handleTransfer(event: Transfer): void { const qlTransfer = new QLTransfer(event.transaction.hash.toHexString()); qlTransfer.from = event.params.from; qlTransfer.to = event.params.to; qlTransfer.tokenId = event.params.tokenId; const contract = NFT_WEB3_EXPOLRER.bind(event.address); // Interact with the contract to obtain tokenId Corresponding tokenURI qlTransfer.tokenURI = contract.tokenURI(event.params.tokenId); qlTransfer.save(); log.info('qlTransfer id is {}', [qlTransfer.id]); // take http Convert to ipfs data const splitstr = qlTransfer.tokenURI.split("/ipfs/"); if(splitstr.length < 2) { return; } const ipfsPath = splitstr[1]; log.info('ipfsPath is {}', [ipfsPath]); // obtain metadata data const data = ipfs.cat(ipfsPath) if (!data) { return; } log.error('data is {}', [data.toString()]); // Convert to json Format const value = json.fromBytes(data); // newly build MetaData structure const meta = new MetaData(qlTransfer.tokenId.toString()); const obj = value.toObject(); if (obj != null) { // analysis name const name = obj.get("name") if (name != null) { meta.name = name.toString(); } // analysis image const image = obj.get("image") if (image != null) { meta.image = image.toString(); } } meta.owner = qlTransfer.to; // data storage meta.save(); log.info('meta id is {}', [meta.id]);}

Compile after the code is written

graph build

Deploy to TheGraph The server

// For the first deployment, you need to set key to grant authorization graph auth --product hosted-service {key}// Deploy graph deploy --product hosted-service EXPLORER-OF-WEB3/nft_web3_explorer_subgraph

TheGraph There will be deployment progress on the official website , It's slow because you have to scan all blocks , After deployment, we have back-end data , Next, the front-end display only needs to request the data .

Front end code implementation

We use apollo This library to help us complete GraphQL Interaction .

 Add dependency npm i @apollo/client graphql

stay uitls New under the directory graphql_utils.js To manage graphql Interaction

import { ApolloClient, InMemoryCache, gql} from "@apollo/client";export const getListData = async () => { // Establishing a connection const client = new ApolloClient({ uri: "thgraph Project address ", cache: new InMemoryCache() }); // according to graphql Form request const data = await client.query({ query: gql` query res { metaDatas{ id name image } }` }); const list = data?.data?.metaDatas; console.log("getListData" + list); return list;}

Get your own NFT

If we add 1 Features , Show your own NFT, Then you only need to specify in the query owenr Just write your own address , As shown below

{ metaDatas(where:{owner:" Address "}) { id name image }}

In the use of TheGrpha after , our NFT The list shows the speed stability control at 500ms about , Greatly improves the user experience , Compared with the direct interaction with the contract, the time consumption is reduced 1 More than ten thousand times .

ending

In this paper, we start from NFT The list shows the phenomenon of slow speed and starts to analyze , Find the unreasonable front-end display process , By introducing TheGraph take NFT List presentation time is reduced 1 More than ten thousand times , There are many similar applications in practice . Due to the need to minimize the storage in Ethereum , We store some data off the chain, and even the storage in Ethereum may not have a proper index , You can only get all the data from the front end and then build the index ( For example, belonging to an address tokenId aggregate ). In this case, we need back-end services to help us index the data on the chain and integrate the data off the chain , Can greatly optimize the user experience .