New Ceramic release: ceramic-one with new Ceramic Recon protocol
The Ceramic protocol has undergone a series of updates over the past few months, all focused on improving performance and scalability, enabling developers to build applications that work better and faster. Today, the core Ceramic team is excited to share these updates with the community by announcing the release of ceramic-one.
About the release
The new release of Ceramic includes a data synchronization protocol called Recon, implemented in Rust. This new implementation of the Ceramic protocol enables data sharing between nodes and allows developers to run multiple nodes that stay in sync and are load balanced. All this facilitates highly available Ceramic deployments and reliable data synchronization.
To utilize the Recon protocol for their applications, developers are provided with a binary called ceramic-one.
This new implementation of the Ceramic protocol offers significant performance and stability improvements. Additionally, this release marks a significant shift in making the Ceramic architecture more robust, allowing the team to iterate on and build new protocols in the future.
The new Recon protocol
Recon is a new data synchronization protocol used for synchronizing stream events in the Ceramic network, implemented on top of libp2p. Stream sets bundle multiple streams together, allowing nodes with a common interest in certain streams to synchronize efficiently.
Before Recon, Ceramic nodes broadcasted updates to streams to every node in the network using a simple libp2p pubsub topic. Due to the single channel, nodes would receive stream event announcements they were not interested in, imposing a significant overhead on every node. Additionally, the network's throughput was limited by bandwidth, which led to either prioritizing high-bandwidth nodes or greatly limiting the network throughput to support low-bandwidth nodes.
Recon provides low to no overhead for nodes with no overlap in interest, while retaining a high probability of receiving the latest events from a stream shortly after any node has the events, without any need for remote connections at query time. By shifting updates from the pubsub channel to a stream set, interested nodes can synchronize without burdening uninterested ones. Stream sets also enable sharding across multiple nodes, allowing synchronization of only sub-ranges, which distributes the storage, indexing, and retrieval workload.
Additionally, nodes need to discover peers with similar interests for synchronization. Recon achieves this through nodes gossiping their interests and maintaining a list of peers' interests, ensuring synchronization with minimal bandwidth. Nodes also avoid sending event announcements to uninterested peers.
Performance and robustness improvements
This release, along with the recent Ceramic Anchor Service (CAS) updates, marks significant scalability improvements. Currently, Ceramic provides a throughput of 250 TPS (transactions per second), more than double the previous throughput of up to 100 TPS before the Recon implementation. This increase in throughput is especially important for applications that handle large amounts of user data and require fast transaction times.
These numbers were measured between two nodes that share the same interest. It’s worth noting that nodes without overlapping interests do not affect each other's throughput. This means that, in theory, the throughput of a ceramic-one node scales horizontally. However, there is still one component that puts an upper limit on this: the CAS, which is operated by 3Box Labs. This service is currently a centralized bottleneck in the protocol, which is why the team’s next goal is Self-Anchoring, allowing any Ceramic-One node to operate completely independently.
This release of Ceramic is also a significant step towards making the Ceramic architecture more robust, enabling the team to iterate on it and build new protocol implementations more easily and quickly.
Getting started with ceramic-one
All new Ceramic developers are recommended to use the ceramic-one to start building on Ceramic. Check out the setup guides on the Ceramic documentation to get started.
Developers, who have been building on Ceramic for a while, are encouraged to migrate their applications to the ceramic-one-based implementation. Check out this migration guide to follow the migration steps.
Share you feedback with us!
We would like to get your feedback on building on Ceramic. Do you have any suggestions or ideas of how the core Ceramic team can improve the implementation of Ceramic? Do you have questions or troubles using the new release or migrating your existing application? Share your thoughts and ideas with us by posting on the Ceramic Community Forum.