Written by James Lefrere
mStable is part of a very fast-moving space — DeFi is hard to keep up with at the best of times! To help with this, mStable has been using The Graph to quickly process and query on-chain data since launch, and it has helped us to deliver Web3 projects.
Compared to the tooling available to developers in 2017, The Graph is a breath of fresh air — it really is easy to get started building a schema and tracking contract data sources.
With that in mind, as a project develops over time, requirements will expand and change; how can you ensure your Subgraphs are understandable, extensible and maintainable going forward? How can it tell a richer story from the events, and answer more questions from developers, integrators and users?
In this post, we’ll explore some of the ways mStable has approached these challenges, and provide some tools and ideas for scalable Subgraph development.
We firstly identified some issues with our subgraphs:
- Codebases were similar, but patterns were not shared
- Schemas started to deviate as incremental changes were made
- Config was defined separately (e.g. contract addresses)
- Automation (e.g. for new deployments) was not trivial
- Not enough questions were being answered by the data
- Common problems on the apps side, caused by a lack of clarity
Next, we thought about some simple steps that could be taken to alleviate these issues:
- Use a monorepo to share configuration and code, and for basic automation
- Provide utilities to abstract away complexity
- Redesign schemas to prioritise clarity
- Begin to align schemas by stitching them together
- Reduce data processing by creating metrics
We used lerna to bring together all of our subgraphs in one repository.
Lerna scripts make it trivial to run a new deployment for multiple subgraphs at once:
# Use a shared configuration file to create a `subgraph.yaml` for each project
# pointing to the mainnet contract addresses
lerna run prepare:mainnet# Run `graph codegen` for all subgraphs with the new contract addresses
lerna run codegen# Run `graph deploy` for all subgraphs – easy!
lerna run deploy:mainnet {your-deploy-key-here}
We created a package to provide utilities and shared schemas for the subgraphs (inspired in part by the Curve Subgraph, developed with Protofire).
Though GraphQL doesn’t explicitly support imports yet (though they have been proposed, and it’s something The Graph are working to move forward), it is possible to use ‘schema stitching’ to achieve something similar. We did this in a simple way by adding ‘import comments’ to the start of the schema:
# import Token
# import Transaction
# import Metric
# import Counter"""
Basket Asset (e.g. DAI for the mUSD basket)
"""
type Basset @entity {
id: ID!
Running codegen will then prepend the selected schemas to a generated schema file, which is used in the Subgraph manifest:
### This file is automatically generated ########### BEGIN IMPORTED DEFINITIONS ########"""
An ERC20-compatible token
"""
type Token @entity {
id: ID!
This composability can move beyond schemas, and extend to mappings. In the below example, we use a set of token utilities to create mappings for ERC20 tokens, where the total supply and other metrics will be tracked without extra boilerplate:
export namespace token {
export function getOrCreate(tokenAddress: Address): TokenEntity {
let id = tokenAddress.toHexString()
let tokenEntity = TokenEntity.load(id) if (tokenEntity == null) {
tokenEntity = new TokenEntity(id) let contract = ERC20Detailed.bind(tokenAddress) let decimals = contract.decimals()
tokenEntity.decimals = decimals tokenEntity.address = tokenAddress
tokenEntity.symbol = contract.symbol()
tokenEntity.name = contract.name() tokenEntity.totalSupply = metrics.getOrCreateWithDecimals(
tokenAddress,
'token.totalSupply',
decimals,
).id
// ...more metrics removed for brevity let totalSupply = contract.totalSupply()
metrics.updateByIdWithDecimals(tokenEntity.totalSupply, totalSupply, decimals) tokenEntity.totalTransfers = counters.getOrCreate(tokenAddress, 'token.totalTransfers').id
tokenEntity.totalMints = counters.getOrCreate(tokenAddress, 'token.totalMints').id
tokenEntity.totalBurns = counters.getOrCreate(tokenAddress, 'token.totalBurns').id tokenEntity.save()
} return tokenEntity as TokenEntity
} export function handleTransfer(event: Transfer): void {
let tokenAddress = event.address
let value = event.params.value counters.increment(tokenAddress, 'token.totalTransfers') if (event.params.from.equals(address.ZERO_ADDRESS)) {
counters.increment(tokenAddress, 'token.totalMints')
metrics.increment(tokenAddress, 'token.totalMinted', value)
metrics.increment(tokenAddress, 'token.totalSupply', value)
} else if (event.params.to.equals(address.ZERO_ADDRESS)) {
counters.increment(tokenAddress, 'token.totalBurns')
metrics.increment(tokenAddress, 'token.totalBurned', value)
metrics.decrement(tokenAddress, 'token.totalSupply', value)
}
}
Simply mapping the Transfer event to token.handleTransfer will take care of tracking the token and a number of metrics (note that this does not include token balances or allowances).
This example also shows two other abstractions: counters and metrics.
type Counter @entity {
id: ID! """
Value of the counter; should be positive
"""
value: BigInt!
}
These entities can be referenced easily by ID, which also makes updating a breeze:
// `inputBasset.totalSwapsAsInput` == ID of the Counter
counters.incrementById(inputBasset.totalSwapsAsInput)
It is also possible to update them without the ID, given an address and known type that makes up the ID:
// `masset` is an address
// Counter ID == `${masset}.totalSwaps`
counters.increment(masset, 'totalSwaps')
The same pattern works for Metrics, which are like a BigInt/BigDecimal boxed up for convenience:
type Metric @entity {
id: ID! """
Exact value of the metric, i.e. in base units as an integer
"""
exact: BigInt! """
Decimals used for the exact value (default: 18)
"""
decimals: Int! """
Simple value of the metric, i.e. the exact value represented as a decimal
"""
simple: BigDecimal!
}
It is also straightforward update these values:
metrics.increment(masset, 'cumulativeMinted', massetUnits)
A benefit of this approach is that it becomes very simple to examine metrics and counters over time with block filters.
Has science gone too far?
savingsContracts {
dailyAPY
}{
"data": {
"savingsContracts": [
{
"dailyAPY": "50.95327849950802"
}
]
}
}
mStable’s SAVE provides a simple and safe way to earn a high interest rate on stablecoins. One of the most significant questions for our data is therefore: what’s the APY?
Previously this has been calculated by making queries over time and computing the result in apps, but this is not ideal:
- More work is being done on the client, and more data is loaded
- Other integrations have to perform the calculation themselves
- It’s difficult or costly to report this data over time
With the design of our Savings Contract, it is possible to do this calculation on The Graph without needing to perform queries on the data (which are not possible in Subgraph event mappings).
This is achieved by creating exchange rate entities when users deposit savings; these represent the rate of savings credits to the underlying collateral. By comparing the timestamps and rates of these entities, it is possible to calculate the APY for a given pair of exchange rates (ideally 24 hours apart). By maintaining references to the last received rate, the rate received 24 hours ago, and, for every rate, the next rate that was received, we walk through the rates in order to find the closest pair:
if (exchangeRate24hAgo == null) {
// Set the first 24h ago value (should only happen once)
savingsContractEntity.exchangeRate24hAgo = exchangeRateLatest.id
savingsContractEntity.save() } else if (exchangeRateLatest.timestamp - exchangeRate24hAgo.timestamp > SECONDS_IN_DAY) { // The '24hAgo' rate should be _at least_ 24h ago; iterate
// through the 'next' rates in order to push this rate forward.
while (exchangeRate24hAgo.next != null) {
let exchangeRateNext = ExchangeRateEntity.load(exchangeRate24hAgo.next) as ExchangeRateEntityif (exchangeRateLatest.timestamp - exchangeRateNext.timestamp > SECONDS_IN_DAY) {
savingsContractEntity.exchangeRate24hAgo = exchangeRate24hAgo.id savingsContractEntity.dailyAPY = calculateAPY(
exchangeRate24hAgo = exchangeRateNext
} else {
break
}
}
exchangeRate24hAgo as ExchangeRateEntity,
exchangeRateLatest,
)
savingsContractEntity.save()
}
This could be made clearer, or optimised with a different design, but it was an interesting case study in how to provide a value for one of the most commonly-requested metrics for mStable; perhaps it can help you to think about what else could be tracked on your contracts to answer more questions.
Going forward, we will concentrate on answering even more questions with these Subgraphs, linking up with other Subgraphs more, and adding better examples and documentation, to make building with mStable easier.
Happy Subgraphing!