In a world of decentralisation, blockchains and dapps, it’s hard to stand out. But that’s exactly what SubQuery managed to do. A key tool that makes it easier for web3 apps (dapps) to work with data.

At its core, SubQuery simplifies the complex process of data indexing and querying.

This way, developers can focus on building user-centric applications without being thrown off by the overly complicated ramifications of blockchain data structures.

The architecture is designed with flexibility and scalability in mind.

The open-source indexer allows us to run their indexing nodes, providing full control over data processing and querying capabilities.

Devs, for us, this component is essential for projects that require customised indexing logic or those that prioritise data sovereignty.

On the other hand, SubQuery's managed service offers a hassle-free alternative. SubQuery’s team takes care of the infrastructure management, allowing developers to concentrate on their application logic.

In essence, SubQuery brings it all together with:

An open-source data indexer.
A managed service solution.
A decentralised network marketplace.

Each of these fulfilling a unique role in levelling up the efficiency and scalability of web3 dapps.

Just take a look at all the (157 and growing) supported networks right here!

Now that went over the basic introduction, let’s dive a little deeper into each component!

Data Indexer

At the heart of SubQuery is its open-source data indexer, a tool designed to be fast, flexible, and reliable.

This indexer serves as the foundation for developers, allowing them to create custom APIs tailored to their specific project needs.

With support for over 157 blockchain networks, developers gain the ability to efficiently access and manipulate blockchain data for their applications.

Since it’s open source, you can check out the SubQuery Github here!

Managed Service

SubQuery's optional managed service solution offers a centralised infrastructure hosting option for developers seeking enhanced capabilities.

This provides features such as dedicated databases, redundant clusters for improved reliability, intelligent multicluster routing, and advanced monitoring and analytics.

The goal is to streamline the development process by offering a robust and managed environment for blockchain data operations, making developers' job easier.

Decentralised network marketplace

The decentralised marketplace within the SubQuery ecosystem operates with the SQT token, providing a unique avenue for participants to engage in buying and selling blockchain data.

This marketplace is a hub for network participants including node operators offering indexing and RPCs services

In other words, SubQuery's comprehensive approach addresses the critical challenge of accessing blockchain data for Web3 applications.

By combining the:

flexibility of an open-source data indexer
the advanced features of a managed service solution
and the decentralised transactional power of a marketplace

SubQuery positions itself as a catalyst for the seamless development and deployment of blockchain-powered applications.

Join the SubQuery community and be part of the movement shaping the future of Web3!

A New Project

Let’s initialise a new project that provides you with a working SubQuery project in your chosen layer-1 network. This will provide a basic understanding of how SubQuery works!

Prerequisites

Before we get to your first blockchain project with SubQuery, we need to make sure you have installed the required supporting software.

NodeJS: A modern (e.g. the LTS version) installation of NodeJS.
Docker: This tutorial will use Docker to run a local version of SubQuery's node.

1. Install the SubQuery CLI

The first step is to install SubQuery globally on your terminal using NPM.

It’s not recommended to use yarn global for installing @subql/cli because of its poor dependancy management. It may lead to multiple errors.

# NPM
npm install -g @subql/cli

# Test that it was installed correctly
subql --help

2. Initialise a new SubQuery Project

Run subql init inside the directory that you want to create a SubQuery project in.

After that, you will be asked a list of questions as you proceed ahead:

Project name: A project name for your SubQuery project.
Network family: The layer-1 blockchain network family that this SubQuery project will index. Use the arrow keys to select from the available options (scroll down as there are multiple pages).
Network: The specific network that this SubQuery project will index. Use the arrow keys to select from the available options (scroll down as there are multiple pages).
Template project: Select a SubQuery template project that will provide a starting point in the development. For some networks we provide multiple examples.
RPC endpoint: Provide an HTTP or websocket URL to a running RPC endpoint, which will be used by default for this project. You can use public endpoints for different networks, your own private dedicated node, or just use the default endpoint. This RPC node must have the entire state of the data that you wish to index, so we recommend an archive node.
Git repository: Provide a Git URL to a repo that this SubQuery project will be hosted in.
Authors: Enter the owner of this SubQuery project here (this can be your name, etc) or accept the provided default.
Description: Provide a short paragraph about your project that describes what data it contains and what users can do with it, or accept the provided default.
Version: Enter a custom version number or use the default (1.0.0).
License: Provide the software license for this project or accept the default (MIT).

Here is an example:

subql init
Project name [subql-starter]: test-subquery-project
? Select a network family Ethereum
? Select a network Ethereum
? Select a template project ethereum-starter     Starter project for Ethereum networks
RPC endpoint: [https://eth.api.onfinality.io/public]:
Git repository [https://github.com/subquery/ethereum-subql-starter]: https://github.com/jamesbayly/test-subquery-project  ^ Author [SubQuery Team]: James Bayly
Description [This project can be use as a starting po...]: A new example ethereum SubQuery project
Version [0.0.1]:
License [MIT]:
Preparing project... done
test-subquery-project is ready

After you complete the initialisation process, you will see a folder with your project name created inside the directory. Please note that the contents of this directory should be near identical to what's listed in the Directory Structure.

Finally, run the following command to install the new project’s dependencies from within the new project's directory.

cd PROJECT_NAME
yarn install

You have now successfully initialised your first SubQuery project in a few simple steps.

Let’s now customise the standard template project for a specific blockchain of interest.

3. Make Changes to Your Project

We should start with modifying 3 important files.

The GraphQL Schema in schema.graphql.
The Project Manifest in project.ts.
The Mapping functions in src/mappings/ directory.

SubQuery supports various blockchain networks and provides a dedicated guide for each of them. Select your preferred blockchain under 2. Specific Chains and continue the quick start guide.

Aurora Quick Start (EVM)

Aurora is a next generation Ethereum compatible blockchain & ecosystem that runs on the NEAR Protocol. This powers the innovations behind Aurora cloud - the fasters path for Web2 businesses to capture the value of Web3.

Since SubQuery fully supports NEAR and Aurora, you can index data from both execution environments in the same SubQuery project and into the same dataset.

In the earlier Quickstart section , you should have taken note of three crucial files.

Please initialise a NEAR Network project. Now, let’s move forward and update these configurations.

Your Project Manifest File

This file is the entry point to your project. It defines most of the details on how SubQuery will index and transform the chain data.

For EVM chains, there are three types of mapping handlers (you can have more than one in each project):

BlockHanders: On each and every block, run a mapping function
TransactionHandlers: On each and every transaction that matches optional filter criteria, run a mapping function
LogHanders: On each and every log that matches optional filter criteria, run a mapping function

As we are indexing all transfers and approvals for the Wrapped NEAR smart contract, the first step is to import the contract abi definition which can be obtained from here.

Copy the entire contract ABI and save it as a file called erc20.abi.json in the /abis directory.

This section in the Project Manifest now imports all the correct definitions and lists the triggers that we look for on the blockchain when indexing.

Since we are indexing all transfers and approvals for the Wrapped NEAR smart contract, you need to update the datasources section as follows:

{
  dataSources: [
    {
      kind: EthereumDatasourceKind.Runtime, // We use ethereum runtime since NEAR Aurora is a layer-2 that is compatible
      startBlock: 42731897, // Block with the first interaction with NEAR https://explorer.aurora.dev/tx/0xc14305c06ef0a271817bb04b02e02d99b3f5f7b584b5ace0dab142777b0782b1
      options: {
        // Must be a key of assets
        abi: "erc20",
        address: "0xC42C30aC6Cc15faC9bD938618BcaA1a1FaE8501d", // this is the contract address for wrapped NEAR https://explorer.aurora.dev/address/0xC42C30aC6Cc15faC9bD938618BcaA1a1FaE8501d
      },
      assets: new Map([["erc20", { file: "./abis/erc20.abi.json" }]]),
      mapping: {
        file: "./dist/index.js",
        handlers: [
          {
            handler: "handleTransaction",
            kind: EthereumHandlerKind.Call, // We use ethereum runtime since NEAR Aurora is a layer-2 that is compatible
            filter: {
              // The function can either be the function fragment or signature
              // function: '0x095ea7b3'
              // function: '0x7ff36ab500000000000000000000000000000000000000000000000000000000'
              function: "approve(address spender, uint256 rawAmount)",
            },
          },
          {
            handler: "handleLog",
            kind: EthereumHandlerKind.Event,
            filter: {
              // address: "0x60781C2586D68229fde47564546784ab3fACA982"
              topics: [
                //Follows standard log filters https://docs.ethers.io/v5/concepts/events/
                "Transfer(address indexed from, address indexed to, uint256 amount)",
              ],
            },
          },
        ],
      },
    },
  ],
}

The above code indicates that you will be running a handleTransaction and handlelog mapping function whenever there is an approve or Transfer log on any transaction from the Wrapped NEAR contract.

Update Your GraphQL Schema File

The schema.graphql file determines the shape of your data from SubQuery due to the mechanism of the GraphQL query language. Hence, updating the GraphQL Schema file is the perfect place to start. It allows you to define your end goal right at the start.

Remove all existing entities and update the schema.graphql file as follows. Here you can see we are indexing block information such as the id and the blockHeight along with addresses such as to, from, owner and spender, along with the contract address and value as well.

type Transaction @entity {
  id: ID! # Transaction hash
  txHash: String
  blockHeight: BigInt
  to: String!
  from: String!
  value: BigInt!
  contractAddress: String!
}

type Approval @entity {
  id: ID! # Transaction hash
  value: BigInt!
  owner: String!
  spender: String!
  contractAddress: String!
}

SubQuery simplifies and ensures type-safety when working with GraphQL entities, smart contracts, events, transactions, and logs.

The SubQuery CLI will generate types based on your project's GraphQL schema and any contract ABIs included in the data sources.

yarn codegen

This action will generate a new directory (or update the existing one) named src/types. Inside this directory, you will find automatically generated entity classes corresponding to each type defined in your schema.graphql.

These classes facilitate type-safe operations for loading, reading, and writing entity fields. You can learn more about this process in the GraphQL Schema section.

It will also generate a class for every contract event, offering convenient access to event parameters, as well as information about the block and transaction from which the event originated.

You can find detailed information on how this is achieved in the EVM Codegen from ABIs section. All of these types are stored in the src/types/abi-interfaces and src/types/contracts directories.

You can import all these types:

import { Approval, Transaction } from "../types";
import {
  ApproveTransaction,
  TransferLog,
} from "../types/abi-interfaces/Erc20Abi";

Check out the GraphQL Schema documentation to get in-depth information on schema.graphql file.

Now that you have made essential changes to the GraphQL Schema file, let’s proceed ahead with the Mapping Function’s configuration.

Add a Mapping Function

Mapping functions define how blockchain data is transformed into the optimised GraphQL entities that we previously defined in the schema.graphql file.

Navigate to the default mapping function in the src/mappings directory. You will be able to see three exported functions: handleBlock, handleLog, and handleTransaction. Replace these functions with the following code:

import { Approval, Transaction } from "../types";
import {
  ApproveTransaction,
  TransferLog,
} from "../types/abi-interfaces/Erc20Abi";
import assert from "assert";

export async function handleLog(log: TransferLog): Promise<void> {
  logger.info(`New transfer transaction log at block ${log.blockNumber}`);
  assert(log.args, "No log.args");
  const transaction = Transaction.create({
    id: log.transactionHash,
    txHash: log.transactionHash,
    blockHeight: BigInt(log.blockNumber),
    to: log.args.to,
    from: log.args.from,
    value: log.args.value.toBigInt(),
    contractAddress: log.address,
  });

  await transaction.save();
}

export async function handleTransaction(tx: ApproveTransaction): Promise<void> {
  logger.info(`New Approval transaction at block ${tx.blockNumber}`);
  assert(tx.args, "No tx.args");

  const approval = Approval.create({
    id: tx.hash,
    owner: tx.from,
    spender: await tx.args[0],
    value: BigInt(await tx.args[1].toString()),
    contractAddress: tx.to,
  });

  await approval.save();
}

The handleTransaction function receives a tx parameter of type ApproveTransaction which includes transaction log data in the payload.

We extract this data and then save this to the store using the .save() function (Note that SubQuery will automatically save this to the database).

Check out our Mappings documentation to get more information on mapping functions.

Build Your Project

Next, build your work to run your new SubQuery project. Run the build command from the project's root directory as given here:

yarn build

Now, you are ready to run your first SubQuery project. Let’s check out the process of running your project in detail.

Whenever you create a new SubQuery Project, first, you must run it locally on your computer and test it and using Docker is the easiest and quickest way to do this.

Run Your Project Locally with Docker

The docker-compose.yml file defines all the configurations that control how a SubQuery node runs. For a new project, which you have just initialised, you won't need to change anything.

However, visit the Running SubQuery Locally to get more information on the file and the settings.

Run the following command under the project directory:

yarn start:docker

Query your Project

Next, let's query our project. Follow these three simple steps to query your SubQuery project:

Open your browser and head to http://localhost:3000.
You will see a GraphQL playground in the browser and the schemas which are ready to query.

Find the Docs tab on the right side of the playground which should open a documentation drawer. This documentation is automatically generated and it helps you find what entities and methods you can query.

Try the following queries to understand how it works for your new SubQuery starter project. Don’t forget to learn more about the GraphQL Query language.

# Write your query or mutation here
{
  query {
    transactions(first: 2, orderBy: BLOCK_HEIGHT_ASC) {
      totalCount
      nodes {
        id
        txHash
        blockHeight
        to
        from
        value
        contractAddress
      }
    }
  }
}

Congratulations! You have now a locally running SubQuery project that accepts GraphQL API requests for transferring data.

The TL;DR

With SubQuery, you can easily access and manipulate blockchain data, as they address the critical need for efficient, scalable, and flexible data indexing and querying solutions.

Having said all that, now is a good time to visit SubQuery and start building!

Blog

👀 A developer’s overview of SubQuery

Deni