Container file system from Portworx goes open source

Portworx, a provider of container data services for DevOps, has announced that it is open-sourcing a filesystem that is purpose-built for containers: the Layer Cloning File System (LCFS). Created to encourage increased innovation in a fundamental technology that boots all containers, LCFS aims to improve the speed of downloading, booting, tearing-down, and building containers.

LCFS operates directly on top of block devices, as opposed to two filesystems that are then merged. The filesystem also directly manages at the container image’s layer level, effectively eliminating the overhead of having a second filesystem that is later merged.

“Our immediate plan is to improve the experience with stateless and stateful containers, but this is just a start,” said Gou Rao, CTO of Portworx. “The future direction is to enhance LCFS with cluster-level operations, richer container statistics, and pave the way toward content integrity of container images.”

Running containers on the same server is often limited by the side effects that come from mapping container behavior over general filesystems. The approach impacts the entire lifecycle: building, launching, reading data, and exiting containers.

While file systems were built with the expectation that content is read/writeable, Docker images are constructed using many read-only layers and a single read-write layer. As more containers are launched off of the same image, reading a file within a container requires traversing all the other containers running that image.

Key advantages:

- Layers are managed directly: LCFS inherently understands layers, their different states, and is able to directly track and manage layers
- Clone independence: The filesystem creates and runs container images as independent entities, from an underlying filesystem perspective. Each new instantiation of the same Docker image is an independent clone, at the read-only layer
- Containers in clusters: It is also very well-suited for clustered operation, optimizes for common data patterns and avoids inheriting behavior that overlaps

Unlike non-filesystem storage drivers, which create multiple copies of the same image layers in memory, LCFS doesn’t cache pages, enabling maximized host memory for containers. For added space management, LCFS improves garbage collection by automatically removing orphaned layers. The storage driver also prevents the creation of multiple inodes per file, effectively avoiding inode exhaustion in build scenarios and at-scale.