Into the Dataverse
A future where everyone in the world is building applications on the same networked, composable data layer.
There has been much hype about the Metaverse, a vision for our digital future that's increasingly merging with Web3's. For this article, let's sidestep attempts by Facebook, ahem Meta, or NFT tribes to stake their claim to the Metaverse. We think of the Metaverse as the entirety of all composable and interoperable resources, identities, applications, platforms, services, and protocols that exist in cyberspace.
Our vision of the Metaverse will run on the Dataverse: a composable, web-scale data ecosystem—owned by everyone and no one. Developers will build interchangeable interfaces and online experiences that directly interact with the Dataverse and the composable data that lives there. The Dataverse will play arguably the most important role within the Metaverse: a shared, high-performance, always-available graph of all data created and used by all applications.
Most features on the web are just data rather than monetary value transactions. For example, the DeFiverse would refer to the universe of interoperable financial protocols, assets, and applications; but, just think about the applications you use on a daily basis. How many times do you click, ‘send’, or ‘like’ compared to ‘buy’ or ‘transfer’? In the Dataverse, your data follows you around from app to app, putting you at the center and in control of your own digital universe.
Over the next five years we will see a massive explosion in the developer market and a full integration of permissionless protocols into the mainstream development stack, giving rise to a billion networked applications that run and interoperate on shared, composable backend infrastructures. That’s an app for every 10 people on the planet.
We’d like to take you on a tour of the Dataverse—how we navigate from the data silos of today’s Web2 apps and the data desert of today’s Web3 apps—to a future where everyone in the world is building applications and they’re all doing it on the same networked, composable data layer.
The Golden Era for Application Development
The route to that interoperable Metaverse depends on enabling developers. The next five years will be the golden age of application development.
Everyone’s an App ‘Developer’
Application development is going to become so accessible that five years from now it will be possible for anyone in the world, regardless of location, technical ability or education level, to build and launch an application on the internet. Basically everyone will be able to be a “developer.” For example, even a non-technical journalism student will be able to launch a fully-featured, crowd-sourced intelligence forum in a few hours.
Bye, bye, backend.
The trend of everything-as-a-service will have abstracted much or all of the backend application development logic away, allowing front-end developers to launch zero-infrastructure applications. These infrastructures could be provided by either decentralized networks or managed service providers.
Open, Shared Platforms
By this time, decentralized permissionless networks will be widely adopted by the mid- and long-tail of application developers, with some forward-leaning Web2 companies joining in on the fun as well. The primary benefit of building on these platforms is that, in addition to providing zero-infrastructure deployments, it makes cooperation and interoperability within the Metaverse feasible. This solves a lot of traditional application development challenges including centralized platform risk (read: Twitter shutters API, Facebook shuts game developers out of its platform), onboarding new users, bootstrapping content, building network effects, creating delightful user experiences, etc.
There are two primary trends driving the minimization of front-end development. First, low/no code solutions are doing a good job of making it easy for non-technical folk to create an app. Second, by nature of building on open platforms and users desiring increased transparency, more application code is getting open sourced than ever before at an accelerating rate. Combined with zero-infrastructure backends this means that basically every application is available to be inspected in its entirety, sampled, forked, cloned, remixed, modified, and redeployed. Creating applications becomes so easy that it looks more like copy/pasting others’ front-end code or dropping modular pieces that work together onto a canvas.
Global Access, Equal Opportunity
Connectivity will have improved around the world through the rollout of satellite-based over-the-top 3G+ internet delivery, causing a new generation of developers to come online for the first time. This connects more people over cyberspace, and gives the inclined more incentive to create something in a global market. The web will be filled with free educational material—all it will take is a little curiosity and initiative.
Powerful Incentives and Direct Rewards
Given two ways of building the same thing, all else equal, people would want to work on the one with the most upside. Web3 provides powerful native participation and builder incentives that are compelling and directly accessible to developers of all backgrounds. Large tech companies will struggle to compete for talent, as the upside potential of going from 0 to 1 on your own (or joining a Web3 rocketship) will far outweigh the draw for working at a company like Amazon. The Metaverse as a whole will outcompete for talent.
Power to the People
The organizing principle of the Metaverse will be the individual. When developers build on this shared application architecture, because it is better for their application (user-centric composability), it coincidentally unlocks a new world for end-users where they can reclaim control over their digital identity and begin to experience the web as if every application was built specifically for them and their data was magically everywhere they wanted it to be.
Self Agency and Determinism
In the Metaverse, individuals have a choice. Based on foundations of public key cryptography, users in the Metaverse begin their life—no permission needed—as a simple, self-sovereign, pseudonymous keypair. From there, they can decide who they want to be. They can allow someone else to control their identity for them. They can be their own bank and take self-custody. They can begin to attach as much or as little data as they want. They can set permissions for who can do what with their data under which conditions. This level of digital ownership and self-agency is how the Metaverse can stay independent and remain secure.
But self-sovereignty doesn’t stop at an account. Users will also have trusted independent infrastructure for keeping their data around for the long-term so they never have to rely on someone else.
The Magic of Interoperability
If one of the primary draws to building in the Metaverse is interoperability, then applications are naturally incentivized to adopt the minimum foundational standards needed to enable this interoperability. The easiest and most intuitive way for interoperability to be achieved is through adopting a user-centric architecture. In this model, the user acts as the nexus of data interoperability between different applications; wherever they go, so does their data.
For end users, the experience of interoperability feels magical—like how the internet should work. No more passwords, forms, verifications, reposting content, refollowing the same person across platforms.
An explosion of Knowledge, Science, and Human Progress
As a public resource, the Dataverse will enable an explosion of shared knowledge and accelerate science and human progress. The original use case for Internet 1.0 was sharing scientific databases between California and Massachusetts. Web 3.0 is going to take that idea and supercharge it.
Until now, semantic web and linked data implementations, which are good for information graphs, have really only been deployed in production within the internal systems of the largest companies and institutions in the world such as Google. They’re hard to build and require many of the brightest minds.
The Dataverse will provide simple, open access to a graph of semantically-defined, structured data that takes the most cutting edge research and allows everyone to build and collaborate on it. While the implications of the progress that can be made when we can connect the entire scientific community on one unified verifiable data platform cannot be overstated, we don’t need to go into unnecessary forward-looking predictions on what that might ultimately look like.
The Creativity of a Billion Networked Applications
What can be known is the diversity, creativity, ingenuity, and innovation that will be unleashed when everyone in the world is building applications and they’re all doing it on the same networked, composable data layer—let’s refer to this network that supports the Dataverse as a platform.
As more developers are joining the platform, and more data exists on the platform, it will become even easier for developers to create applications. The startup costs drop to near zero. This allows for a diversity of implementations and experiments that add resilience and allow Darwinism to chart the path forward. It allows developers to hack scalable microapplications together in a few minutes and focus on product-market fit instead of taking weeks to ship a hacky monolith. It provides a new vector to allow everyone-but-Facebook-or-Google to compete with Facebook and Google. The pace and ingenuity of the bazaar, in this case, will outpace the cathedrals.
A few trends that define the Dataverse and the future of the web.
Web3 Applications Explode With Data Functionality
The Web3 developer market, containing collaborative groups of value-aligned innovators/early adopters (some of which are already building in other parts of the Metaverse), will be first to adopt the Dataverse. To date, Web3 applications have been data deserts. This makes sense, as there’s only so much data functionality you can build with smart contracts.
This trend is changing and will continue at an accelerating rate as the market becomes more crowded/mature and applications look for new vistas for innovation. As the market demand picks up for building data features and applications begin cooperating over large data sets, the network effects of the Dataverse will quickly accelerate and the Web3 application market will be flush with data-powered features. Others beyond this market will notice the ecosystem and be compelled to join it.
Interoperable Ecosystems Will Outcompete Siloed Platforms
The perfect cocktail is brewing for the Metaverse to outcompete even the largest, most entrenched institutions. No one will want to build their own application on an island anymore or for big companies. The simplicity, creativity, pace of innovation, speed of development, open data, and rewards that are available in the Metaverse will be too strong. The incumbents who have already built their business on an entirely different incentive structure will find it tough to pivot or compete.
Identity Emerges From Needs of Communities and is the Organizing Principle of the Dataverse
Identity is inherently multiplayer. This is true not only in the physical world, where it began as a construct enabling human-to-human interactions, but also in the digital world where it’s more important to have a profile on a social media application than it is to have on your personal note-taking application. Identity is a tool for providing context, trust, and efficiency to transactions. By providing an inherently multi-player data ecosystem, Ceramic is a platform that we can expect to generate a widely-adopted identity system that can serve to provide greater interoperability to the Metaverse.
The Effectiveness and Sustainability of Digital Communities
Less technical and more sustainability related, we’re betting on the fact that strangers on the internet, unified by a community-owned protocol and its inherent incentives, can effectively organize and create high-impact work with high efficiency. Although this trend has positive indications, it still stands to be seen if these communities can sustain themselves over the long term or if they can stay innovative and keep speed high. The design and implementation of effective digital communities is something we’re keeping a close eye on, but we also believe that a lot of what will make communities successful over the long term has yet to be discovered.
Some people have gone full degen into “working for the internet” as a contributor to protocols and DAOs, picking up one-off tasks or engaging in longer contracts. We already have participants in DAOs lobbying in front of congress. DAOs are legally recognized meatspace entities in Wyoming. Digital communities have long been used for OSINT (open-source intelligence), where groups of highly-motivated strangers come together on topic-based platforms such as subreddits or Facebook groups to do things that any single person (or government agency) couldn’t—such as crowdsourcing villains or solving investigations. Wikipedia runs beautifully. These are all things that make me optimistic about how this fluid work and incentive system will evolve over time.
There are positive indications that these communities are beginning to cross the chasm into mainstream, which only deepens their tentacles into society making them more resilient. Bored Ape Yacht Club basically has every celebrity with an ape as their Twitter profile picture; Alexis Ohanian and Serena Williams wore pins of their CryptoPunks to the Met Gala. These digital-first communities will spill into every aspect of society with influencers and celebrities sporting their flair and representing their tribe.
To get a sense of where Web3 (as opposed to NFTs) will cross into mainstream culture, we should look at where there exist networks of highly-engaged, multiplayer user bases/communities, what it means to be a member of that community, and how people fly their flag.
Ceramic: Decentralized Infrastructure for Building on the Dataverse
Although the market may be primed for the Dataverse, it doesn’t necessarily mean it will emerge organically. The best developer platform for building applications in the Dataverse requires a few key design considerations.
General-Purpose Data Protocol
Data is highly varied, and developers need freedom and flexibility to build their applications and data use cases. Rigid standards won't work, or will lead to fragmentation. We need a highly-generalized protocol that is capable of supporting the diversity of use cases that will emerge from the Dataverse. It’s important that all these use cases seamlessly interact and the easiest way is to have them building on the same protocols. It shouldn’t matter which wallet users log in with or what feature you’re building on the front end or what type of data you want to store; it should all just work.
Fast, Decentralized, High-Volume Processing
This general purpose data protocol needs to support mutability with fast, decentralized, high-volume transaction processing simply for the fact that there are so many data events occurring all day, every day, in every application. Current blockchain protocols cannot scale to support this level of processing and this is a strong area of differentiation. Data consensus properties, like Ceramic’s, can be treated differently than value consensus properties, like Bitcoin’s.
Globally-Consistent, Always-Available State
Next, it’s important that the system has a way to provide participants with a highly available state that can be trustlessly verified and reconstructed from the underlying event streams. This is critical, because it’s actually what allows developers to build permissionless, composable applications on shared data. The availability of state can be ensured by anyone who plugs into the underlying event streams, as updates are published through this system all participants can immediately update their state database.
Related to global state availability is state representation. Ceramic currently supports state outputs in the form of simple JSON objects and a native indexing system, which is good for graph-based application use cases. The protocol should be able to adapt and support different state representations based on the optimizations required by the data structures better suited to these use cases. This can be achieved by plugging into the event streaming layer of Ceramic and building custom databases on top.
Every piece of data in the Dataverse should be verifiable, secure, and useful—not spam or bits of data that clog up the network. By using a protocol for decentralized, cryptographic, object-oriented permissions, we can give decentralized identities provable agency over their data while allowing the access permissions for each state (or even nested/recursive/chained permissions to groups of states) to be completely customizable.
For example, users can grant their therapist, their school counselor, and their best friend permanent access to their diary. Or someone can give an application the ability to update a state on their behalf for the duration of a session. It can also be natively integrated with the different cryptographic authentication systems used within developers’ applications (e.g. sign in with Ethereum). It’s important that we collaborate with other Dataverse protocols and services to adopt the same permissioning standard so that we can achieve maximal interoperability. In a lot of ways, the more robust, developed, and flexible the permissions protocol is, the less important it is to tie an identity to any single key.
We should also expect a lot of innovation from the developer ecosystem in the area of permissioning writes and reads within the network. We’re already seeing experimentation along these lines with programmatic permissions protocols. Examples include hosted custodial agents or “personal data managers” or something like “credit bureaus” for managing permissions for users’ data. We should encourage all kinds of diversity, especially early on.
Fast, Expressive, Network-Wide Queries
Whenever a user logs into an application with their wallet, that application needs to gather data about that user from the Dataverse. When an application wants to fetch a single state that they already somehow know the identifier of—that’s easy. But application developers need much more data retrieval capability than fetching content using its specific machine-readable identifier.
When this application is multi-player, such as a social network, developers need to gather data sets from across users to display in one interface. This type of data retrieval functionality is core to almost any application, and is especially core to the ones that are most valuable to the data ecosystem as a whole. When the identifiers of the users are known, this can be done by looking at each user’s index of their data that exists in the Dataverse and then pulling in that data. This supposes the existence of a protocol for such a user-centric index, whether physically stored as a single list somewhere in the Dataverse, or maintained using metadata and an indexing service.
Applications, however, don’t always have the full picture about a user—a wallet address alone doesn't tell the full story. Context can be gained by looking at the broader social graph of communities where the user participated in or rankings in games.
Developers who build on Ceramic will have the ability to use indexing and querying to pull in opt-in data from users in other apps in the Ceramic ecosystem.
Ceramic will soon offer a graph database that will enable developers to explore those ties from any of their user's accounts to a stream of data, e.g. all blog posts. But in further iterations, Ceramic will allow relationships between any account, model, and data stream related to model. This will allow infinitely complex queries across the Dataverse.
Open Marketplace of Data Models
The Dataverse demands interoperability. This is achieved with emergent consensus on data models. Data models on Ceramic are similar to ERC contract standards in Ethereum—e.g. ERC20 for tokens, ERC721 for NFTs—which are basically the equivalent of features for Ceramic. This makes it easy for any developer to just choose from a pre-existing set of feature templates, add a few lines of code to their frontend, and have an app that can access all data sets in the Dataverse that also conform to the same data model.
The community can build out and curate the biggest universe of data models in the world. The most powerful way to create enough good standards for this to become a vibrant marketplace is by giving the community ownership and upside.
The Dataverse is owned by no one, and everyone. It’s the output of a community all working toward building creative, networked applications that result in creating a valuable resource for the world. This big project can only be successful if there is a large, engaged community behind it.
Community members can help with building out the developer tooling and stack ecosystem, growth and influencing, creating educational content, network operations, etc. We have to deliberately consider the incentives of all ecosystem participants so we can maximally deploy this community for good.
We will start by focusing on delivering data-rich application capabilities to advanced Web3 developers building in highly-networked application ecosystems using data models—the market and use cases most primed for Ceramic. Each phase will look to the previous for a signal on what infrastructure to use and how to build, that’s why full focus on today’s innovators is so important.
Unmet Needs of Web3 Developers
The Metaverse is emerging from the foundation paved by blockchain; it possesses all the right conditions mentioned earlier to make it a flourishing, vibrant interoperable application ecosystem. Web3 developers are already building applications on blockchain, and a lot of them are looking to differentiate and mature their product by expanding into data.
Reputation features are needed across many different Web3 multi-player applications, decentralized protocols, social communities, and span everything from a profile, to assets I can prove to own (NFTs, domain names, social handles), to experience (self-asserted and attested skills trees), to contribution tracking, to KYC VCs (verifiable credentials).
Some of these use cases include VCs, many do not. Within this set, Ceramic can serve public and ephemerally private data use cases. Fully private use cases should be stored encrypted on an access-controlled server that ideally uses IPLD and CACAO so it can natively interoperate with the data on Ceramic—something like Kepler from Spruce Systems.
Over the longer term, this collection of data sets will function like one’s identity and passport within the Metaverse. The stakes are high. For example, the future of DeFi is very much in flux until reputation and credentials are figured out.
Social is very hot now in Web3 and a lot of momentum is building around the decentralized social graph and decentralized social media. We see an expansion of the traction and experiments from Orbis, Convo, and CyberConnect, into different iterations and experimentations. These may come in the form of full-blown decentralized social networks, or social features embedded within other applications. Expect social graphs, feeds and posts (blogs, microblogs, etc), reactions, private and public groups, DMs, forums, address books.
Many applications and protocols are asking themselves how to identify a user when they have many different wallets or accounts. This is becoming more important for Web3 applications that are integrating more and more Metaverse protocols into a single application. Maintaining that user object with all the mappings becomes cumbersome and is extremely prone to implementation error. There are a certain set of data features that can be explored for different models of creating an aggregated identity for users in the multi-account, cross-chain (MACC) Metaverse.
The Best Has Yet to Come
The best has yet to come for the internet. The Metaverse will be the biggest breakthrough in application development over the last 25 years, and it will replace the cloud as the application environment of choice.
In order to unlock the full potential of the Dataverse, there needs to be interoperability. This type of composable, interoperable data network will be the catalyst for a new wave of innovation that not only reshapes the way we build applications but also how we share knowledge, invent science, and progress as a society. This is the world we are building toward with Ceramic.
To start building with Ceramic, check out our guide or join the Discord!