Connecting Rust and IPFS

Introduction

The InterPlanetary File System (IPFS) is a decentralized system for sharing data across the internet. IPFS allows its users to send and receive data directly with each other, similar to BitTorrent. Participants act as both servers and clients on the IPFS network, which implements a number of improvements to previous peer-to-peer sharing technologies, such as data de-duplication and replication.

IPFS is positioned to become the storage layer of the decentralized web, and is already in production use by industry leaders. Protocol Labs maintains a reference implementation of IPFS written in Go.

Opportunity

The goal of this project is to offer a full-featured, secure implementation of IPFS in Rust. We use the reference test suite¹ as our success metric. This effort involves comprehensive project planning, open-source community engagement, and expert Rust development skills.

Timeline

Late 2019: Opportunity identified Equilibrium identified community requests for a Rust implementation of IPFS in late 2019 and initiated a thorough review of the existing IPFS codebase and protocol specifications. This analysis informed the project roadmap: we outlined the required work phases and development milestones, prioritizing the early release of core features.

Early 2020: Phase 1 We released several core features, fostered community engagement and began outreach to Protocol Labs to explore grant opportunities.

Mid 2020: Phase 2 We completed additional networking security and performance improvements throughout 2020, enabled related technologies such as offchain::ipfs², implemented UnixFS, and expanded conformance test coverage.

Late 2020 through 2021 Development continues on the remaining IPFS features, as well as exploring new opportunities, such as utilizing Tokio³ for improved networking capabilities.

Progress Achieved

We maintain a website, Are We IPFS Yet, where we track progress towards feature-parity with the reference implementation and highlight community engagement opportunities. We have made significant progress towards full compliance with the reference test suite. Each of these features is now fully supported in Rust:

• Pubsub
• Swarm
• Refs
• Cat
• Get

Development of additional components is currently underway.

Impact Review

Open-Source Project Guidance We are fostering an active, mature Github community to support the Rust IPFS codebase, bringing together development resources from projects that had languished for years prior.

Vision Equilibrium recognizes IPFS as a key technology supporting the distributed web. We believe developing IPFS in Rust offers more robust security and higher performance, as well as bringing IPFS to more platforms, such as embedded systems, IoT and low-power devices. Rust's safety mechanisms reduce the occurrence of common errors and vulnerabilities, which allow developers to spend more time on features and performance.

Best Practices Equilibrium sets projects up for long-term success by applying industry best practices. Our comprehensive documentation efforts include not only technical reference materials for seasoned developers, but also quick start guides and tutorials for those who are new to the project. The codebase includes a comprehensive set of software tests to ensure code contributions meet the technical specifications.

About Rust

Rust is a modern open-source programming language providing memory safety and performance comparable to C++, making it a popular choice for developing robust, secure applications. Rust powers projects such as ZCash Zebra⁴, Holochain⁵, Lighthouse⁶ and many others in the blockchain space and beyond.

About Equilibrium

Equilibrium designs and builds core infrastructure for the decentralized web. We love solving hard engineering problems in decentralised technologies to bring projects to production.

Footnotes