Impact Evaluators on Ceramic: Apply for the Grant

Joel Thorstensson -

This article explains how a programmable framework of impact evaluators can be built on top of Ceramic in order to distribute rewards to a wide range of community contributors—allowing DAOs to distribute funds more effectively to the people that create meaningful value.

What is an impact evaluator?

Impact evaluators (IE) are not new. The Bitcoin block reward is likely the simplest, and oldest, example of an impact evaluator. It’s an objective function that takes a measure of the world and distributes rewards based on that measure. Juan Benet defines an impact evaluator as follows:

ImpactEvaluator {
  foreach round {
    m := measure(world)
    w := evaluate(m)
    r := reward(w)
  }
}

The IE runs iteratively in rounds, with each round distributing a reward. There isn’t really a limit as to what the IE can measure and reward. However, it’s important that the measure is fairly objective and as non-gameable as possible. The more objective and immutable the IE is the more powerful it can generally become since actors interacting with it can reason about rewards long into the future.

Examples in Practice

  • Bitcoin block reward: Rewards miners based in minerHashRate/totalHashRate
  • Filecoin block reward: Rewards miners based on storage capacity they add to the network
  • RAI PID controller: Rewards all participants based on the market rate of the RAI token
  • Sourcecred: Rewards contributors on Github, Discord, etc. based on a weighed page rank algorithm
  • Coordinape: Rewards contributors based on the rating and feedback of other contributors
  • Praise by GivEth: Rewards contributors based on how much “praise” they received
  • OpenQ: Rewards developers for closing PRs
  • Just One Giant Lab: Community peer reviews

Govrn

Govrn stands out, from other projects tracking contributions in a social graph, since it allows projects to create an ontology of various types of contributions, which lets the IE of the DAO dynamically evolve over time.

Three Modalities of Rewards

There are thee main methods of distributing the rewards of an IE.

Rewards From DAO Treasuries

The r of the IE comes directly from the DAO treasury that is operating the IE. This can be seen as a way to reward the community of the DAO more effectively than running a grant program, or as a more fair continuous airdrop.

New Token

With a new token, an IE can mimic the Bitcoin reward curve. This can be seen as a fair launch based on the evaluation criteria of the IE.

Governance Shares

In certain DAO structures, such as Moloch, members hold non-transferable shares. An IE could easily be made to distribute these sort of shares as well. Here a Bitcoin-like reward curve could be used as well, and could be customized for the DAO in question.

Verifiable Data on Ceramic

Data written to Ceramic (e.g. using ComposeDB, Ceramic’s decentralized graph database) is ideal for creating verifiable contribution graphs. This is due to three main properties:

  • Hash linked event logs - all data is stored as streams in Ceramic. A stream is an event log that can be independently verified by anyone who runs a node in the network
  • Timestamps - all streams are periodically anchored into the blockchain, providing a tamper-proof timestamp of when data was created
  • Attributable - all streams have one account as the controller and all events added to a stream needs to be signed by this controller. In practice the controller is a blockchain account (e.g. Ethereum address)

Trust Graphs and Evaluations

The framework for IE’s should be programmable such that developers can create their own IE for whatever actions they want to reward their community for. In order to reason about what this programming framework would look like we need to consider what data goes into the evaluation.

Data Apps on Ceramic

Some example of DAO-centric apps that can be built on Ceramic’s data layer:

  • Issue tracking - who opened and closed issues
  • Project management - planning and management of projects
  • Personal contribution logs - individuals can state all the contributions they have made and to which projects / DAOs the contribution might be valuable for (this is the approach MetaCred is using)
  • Governance forums - proposals, discussions, etc.
  • Contribution evaluation - data similar to Coordinape or Praise
  • Knowledge graphs - e.g. Lateral’s Discourse Graph, or Foresight institute’s tech trees

Computation Over the Contribution DAG

All data in Ceramic is represented as an IPLD DAG. An IE would have a specific CID (content identifier) hash that represents the entire contribution graph and runs the computation over this graph. This will make the computation easily reproducible, as long as it is deterministic.

Trusted Seed

In order to mitigate the issue of sybil accounts adding random or noisy data, an initial set of accounts that the graph is based upon is likely a good idea. This can be a simple list at first, but it should eventually leverage something like Gitcoin Passport. In addition to the initial trusted seed, the set of eligible accounts could grow through attestations according to some web-of-trust-like logic in the IE.

Evaluation Function

Let’s explore what the IE function will run each round.

World

First we need to measure the world. We can define a completely verifiable world as follows:

  • Ceramic Data Models - the IE model would select a set of Ceramic models from which to read contribution data
  • Trusted seed - the set of accounts that root the trust of the contribution graph
  • Previously distributed rewards - the rewards that have been distributed in previous rounds
  • Other data - potentially including data from other sources, such as on-chain state

Evaluator

The evaluator itself should be a function with the world as parameters, and return a reward as a list of accounts and value. This function should be easy to implement in a general programming language. Something like Javascript (using Secure Ecmascript) or WASM would make sense.

The world input parameters could be defined as follows:

ImpactEvaluatorParams {
  ceramicData: [CID] // a list of CIDs defining the contribution graph DAG
  trustedSeed: [DID] // a list of initially trusted accounts
  previousRewards: CID // a merkle root of previous rewards
  otherData: CID // other verifiable data, e.g. ethereum state
}

The output of the IE, ideally, is easily claimable on-chain. It could be a CSV file for distributing tokens in Safe. These sort of distributions can be quite expensive as the contribution set grows. A better approach is to use some sort of merkle drop, basically putting the rewards in a merkle tree and allowing people to claim rewards as they see fit. AstroDrop is a good example of this that allows you to easily update the merkle tree. A merkle tree can also be represented in IPLD, which is nice for our compute function. The return value would likely need to return an updated trusted seed, as well, to enable growing of the user set using some sort of web-of-trust model.

ImpactEvaluatorResult {
  rewards: CID // merkle root of rewards
  newTrustedSeed: [DID] // an updated list of trusted accounts
}

Ideally subsets of the IE function can be reused as modules and are composable.

Bringing Results On-Chain

The result of the IE needs to be put on-chain every round so that participants can claim rewards. There are a few different approaches to this outlined below.

  • Oracles - A trusted third party, a set of nodes, or some sophisticated oracle network could be used to put the merkle tree root on-chain. Ideally there is some dispute mechanism that can be used if the oracle misbehaves. Worth noting is that the IE function should be reproducible / deterministic. This means that it’s straight forward for an outside observer to determine if the oracle behaved well or not
  • Verification game - A verification game like fraud proofs in optimistic rollups or Truebit. These can be quite complicated to implement but possible as a way to reduce trust assumptions
  • ZKP - In principle the entire IE function can be evaluated in a zk circuit and the proof posted on-chain. This would make the IE’s completely trustless. However, it would be a quite big undertaking to implement provers for every IE. It would be more convenient to rely on some sort of general circuit. Something like zkWasm or Lurk could be good candidates

A Practical Approach

In order to bring the programmable IE primitive to as many DAOs as possible a good first approach would be an integration with Safe using Zodiac’s reality module. Reality is a configurable oracle that uses Reality.eth. It could be easily used to put the root of the reward merkle tree on-chain, attached to some custom logic. In practice, every round some off-chain actor would load all Ceramic streams and associated data, run the IE function over this data (e.g. using Bacalhau), and make a call to the Reality.eth oracle on-chain. Other actors can run the same computation over the same data and dispute the result if necessary. Final arbitration could be handled by the Kleros court, for example. Note that, in contrast to normal arbitrations that Kleros deals with, here we have a completely objective outcome that the arbiters will need to judge. They can easily run the computation for themselves to see the desired result.

Next steps