Storing authenticated and encrypted data on IPFS is a core building block for many Web3 applications, but to date there has not been a standardized way to encode this type of data.

Without a standard, many developers have been forced to create custom formats for their signed and encrypted data. This has been prohibitive to the openness and interoperability of information stored in IPFS by siloing data to their particular implementation. Another approach to authenticated data has been to put data in IPFS and put the CID of the data in a smart contract on a blockchain, such as Ethereum. This is essentially an expensive way of adding a signature on top of the data and persisting the signature record on the blockchain.

With the introduction of EIP-2844, a standard that allows wallets to support a few new methods for signing and decrypting data based on DIDs and the dag-jose IPLD codec, we can now simply put authenticated and encrypted data directly into IPFS. In this tutorial, you will learn how you can utilize these primitives with two libraries, js-did and 3ID Connect!

What are DIDs and JOSE?

DIDs is the W3C standard for Decentralized Identifiers. It specifies a general way of going from a string identifier, e.g. did:3:bafy..., to a DID document which contains public keys for signature verification and key exchange. In most DID methods the document can be updated when keys are rotated for security reasons.

JOSE is a standard from IETF which stands for JSON Object Signing and Encryption, and that pretty much explains what it is. There are two main primitives in this standard: JWS (JSON Web Signatures) and JWE (JSON Web Encryption). Both of these formats allow for multiple participants: in JWS there can be one or multiple signatures over the payload, and in JWE there might be one or multiple recipients for the encrypted cleartext.

Building with dag-jose and EIP2844

As we have been building out Ceramic with dag-jose and EIP-2844 as basic building blocks, we've created a few lower-level tools which allow us to more easily use these technologies. This tutorial will show you how to use these powerful tools directly.

IdentityWallet is an implementation of EIP-2844 using 3ID as the DID method. It can be used standalone as a DID Provider, or more conveniently within the 3ID Connect library. 3ID Connect allows users to use their Ethereum wallet (support for more blockchains coming soon) to get access to a DID Provider.

js-did is a library that allows developers to represent a user in the form of a DID. This is the main interface we're going to be looking at in this tutorial. It allows us to sign data with the currently authenticated user, encrypt data to any user (DID), and decrypt data with the currently authenticated user.

Signed data in IPFS

By using the dag-jose IPLD codec we can create data structures that are linked and signed. This is done by creating JSON Web Signatures (JWS) that contain a link to additional data. One of the main problems that the dag-jose codec solves is that the payload of a JWS is traditionally encoded as base64url which means that if it contains any IPLD links you can't traverse those links. Instead what we do with DagJWS is enforce the payload to be the bytes of a CID. The codec then transforms the payload into a CID instance and sets it to the link property of the DagJWS. This allows us to easily traverse the resulting DAG.

Setup your environment

This section will cover how to set up some specific dependencies needed for this tutorial. If you just want to skip the setup part we have prepared a simple playground which bundles ipfs, 3id-connect, and dids. You can use it by opening the web page, clicking on connect, then opening the developer console where you can run the commands. If you decide to do so, skip the following two sections.

Setup IPFS with dag-jose support

Since dag-jose is a new IPLD codec it's not yet included in js-ipfs by default. It also implements the new IPLD codec API which is also not supported by js-ipfs yet. Therefore we need to do the following when we are creating an instance of IPFS:

import IPFS from 'ipfs'
import dagJose from 'dag-jose'
import multiformats from 'multiformats/basics'
import legacy from 'multiformats/legacy'

multiformats.multicodec.add(dagJose)
const dagJoseFormat = legacy(multiformats, dagJose.name)

const ipfs = await Ipfs.create({ ipld: { formats: [dagJoseFormat] } })

Make sure to install the correct version of multiformats:

$ npm i multiformats@3.0.3

Setup DID and 3ID Connect

In the example setup below we use an injected Ethereum provider (such as MetaMask) to create a 3ID Connect and DID instance.

import { DID } from 'dids'
import { ThreeIdConnect, EthereumAuthProvider } from '3id-connect'

// create 3id connect instance
const addresses = await window.ethereum.enable()
const authProvider = new EthereumAuthProvider(window.ethereum, addresses[0])
await threeIdConnect.connect(authProvider)

// create did instance
const didProvider = await threeIdConnect.getDidProvider()
const did = new DID({ provider: didProvider })
await did.authenticate()
window.did = did
console.log('Connected with DID:', did.id)

Create a signed data structure

We can now start signing and adding data to IPFS! First lets create a simple function that takes a payload, signs it using the did.createDagJWS method, and adds the resulting data to IPFS. As we can see in the code below we get two objects back from this method: jws which is the DagJWS itself and linkedBlock which is the raw bytes of the encoded payload. What happens in the background is that the payload gets encoded using dag-cbor, after this the CID of the encoded payload is used as the payload of the created jws. We can access this payload CID on the DagJWS instance as jws.link.

async function addSignedObject(payload) {
  // sign the payload as dag-jose
  const { jws, linkedBlock } = await did.createDagJWS(payload)
  // put the JWS into the ipfs dag
  const jwsCid = await ipfs.dag.put(jws, { format: 'dag-jose', hashAlg: 'sha2-256' })
  // put the payload into the ipfs dag
  await ipfs.block.put(linkedBlock, { cid: jws.link })
  return jwsCid
}

Using this function, let's create our first signed data objects:

// Create our first signed object
const cid1 = await addSignedObject({ hello: 'world' })

// Log the DagJWS:
console.log((await ipfs.dag.get(cid1)).value)
// > {
// >   payload: "AXESIHhRlyKdyLsRUpRdpY4jSPfiee7e0GzCynNtDoeYWLUB",
// >   signatures: [{
// >     signature: "h7bHmTaBGza_QlFRI9LBfgB3Nw0m7hLzwMm4nLvcR3n9sHKRoCrY0soWnDbmuG7jfVgx4rYkjJohDuMNgbTpEQ",
// >     protected: "eyJraWQiOiJkaWQ6MzpiYWdjcWNlcmFza3hxeng0N2l2b2tqcW9md295dXliMjN0aWFlcGRyYXpxNXJsem4yaHg3a215YWN6d29hP3ZlcnNpb24taWQ9MCNrV01YTU1xazVXc290UW0iLCJhbGciOiJFUzI1NksifQ"
// >   }],
// >   link: CID(bafyreidykglsfhoixmivffc5uwhcgshx4j465xwqntbmu43nb2dzqwfvae)
// > }

// Log the payload:
ipfs.dag.get(cid1, { path: '/link' }).then(b => console.log(b.value))
// > { hello: 'world' }

// Create another signed object that links to the previous one
const cid2 = addSignedObject({ hello: 'getting the hang of this', prev: cid1 })

// Log the new payload:
ipfs.dag.get(cid2, { path: '/link' }).then(b => console.log(b.value))
// > {
// >   hello: 'getting the hang of this'
// >   prev: CID(bagcqcerappi42sb4uyrjkhhakqvkiaibkl4pfnwpyt53xkmsbkns4y33ljzq)
// > }

// Log the old payload:
ipfs.dag.get(cid2, { path: '/link/prev/link' }).then(b => console.log(b.value))
// > { hello: 'world' }

Note that the values of the CIDs and JWS will be different for you since the payload will be signed by your DID.

Verify a signed data structure

Verifying a JWS is very straight forward. Simply retrieve the JWS object and pass it to the verifyJWS method. If the signature is invalid, this function will throw an error. If the signature is valid, it will will return the DID (with key fragment) that was used to sign the JWS.

const jws1 = await ipfs.dag.get(cid1)
const jws2 = await ipfs.dag.get(cid2)

const signingDID1 = await did.verifyJWS(jws1)
await did.verifyJWS(jws2)

Encrypted data in IPFS

Signed data in IPFS is one piece of the puzzle, but perhaps more interesting is encrypted data. With the use of dag-jose and EIP-2844 we can encrypt data to one or multiple DIDs and store it directly in IPFS. Below we demonstrate how to use the convenient tools provided by the js-did library to do this.

Encrypt IPLD data

There is a simple method to create a DagJWE object which is encrypted to one or multiple DIDs, createDagJWE. This method accepts an IPLD object (a JSON object that may also include CID links) and an array of DIDs. It will resolve the DIDs to retrieve the public encryption keys found in their DID document and create a JWE that is encrypted to these keys. To get going, let's create a helper function that creates a JWE and puts it into IPFS.

async function addEncryptedObject(cleartext, dids) {
    const jwe = await did.createDagJWE(cleartext, dids)
    return ipfs.dag.put(jwe, { format: 'dag-jose', hashAlg: 'sha2-256' })
}

Once we have this function we can create a few encrypted objects. In the example below we first create a simple encrypted object, then we create an additional encrypted object that links to the previous one.

const cid3 = await addEncryptedObject({ hello: 'secret' }, [did.id])

const cid4 = await addEncryptedObject({ hello: 'cool!', prev: cid3 }, [did.id])

Note that in the example above we use [did.id](<http://did.id>) to encrypt the data to the currently authenticated DID. We can of course also encrypt the data to the DID of a user that is not locally authenticated, such as another user!

Decrypt IPLD data

When the data is retrieved from IPFS we will just get the encrypted JWE. This means that we need to decrypt the data after we fetch it. Since we have created objects that link to each other, lets create a function that retrieves these objects and decrypts them recursively.

async function followSecretPath(cid) {
    const jwe = (await ipfs.dag.get(cid)).value
    const cleartext = await did.decryptDagJWE(jwe)
    console.log(cleartext)
    if (cleartext.prev) {
        followSecretPath(cleartext.prev)
    }
}

The function above is a toy example that just logs the decrypted objects. We can use it to look at the content of these objects.

// Retrieve a single object
followSecretPath(cid3)
// > { hello: 'secret' }

// Retrive multiple linked objects
followSecretPath(cid4)
// > { hello: 'cool!', path: CID(bagcqceraqittnizulygv6qldqgezp3siy2o5vpg66n7wms3vhffvyc7pu7ba) }
// > { hello: 'secret' }

That’s it for this tutorial! Hope you enjoyed diving deep into IPFS-enabled authenticated and encrypted data. I'm really looking forward to what you build on top of these powerful standards.

If you want to go deeper into how we are using these technologies in Ceramic, have a look at the js-ceramic Github repo and jump into our Discord.