Container file system from Portworx goes open source
Wednesday, February 15, 2017
|
Richard Harris |
Portworx, a provider of container data services for DevOps, has announced that it is open-sourcing a filesystem that is purpose-built for containers: the Layer Cloning File System (LCFS). Created to encourage increased innovation in a fundamental technology that boots all containers, LCFS aims to improve the speed of downloading, booting, tearing-down, and building containers.
LCFS operates directly on top of block devices, as opposed to two filesystems that are then merged. The filesystem also directly manages at the container image’s layer level, effectively eliminating the overhead of having a second filesystem that is later merged.
“Our immediate plan is to improve the experience with stateless and stateful containers, but this is just a start,” said Gou Rao, CTO of Portworx. “The future direction is to enhance LCFS with cluster-level operations, richer container statistics, and pave the way toward content integrity of container images.”
Running containers on the same server is often limited by the side effects that come from mapping container behavior over general filesystems. The approach impacts the entire lifecycle: building, launching, reading data, and exiting containers.
While file systems were built with the expectation that content is read/writeable, Docker images are constructed using many read-only layers and a single read-write layer. As more containers are launched off of the same image, reading a file within a container requires traversing all the other containers running that image.
- Layers are managed directly: LCFS inherently understands layers, their different states, and is able to directly track and manage layers
- Clone independence: The filesystem creates and runs container images as independent entities, from an underlying filesystem perspective. Each new instantiation of the same Docker image is an independent clone, at the read-only layer
- Containers in clusters: It is also very well-suited for clustered operation, optimizes for common data patterns and avoids inheriting behavior that overlaps
Unlike non-filesystem storage drivers, which create multiple copies of the same image layers in memory, LCFS doesn’t cache pages, enabling maximized host memory for containers. For added space management, LCFS improves garbage collection by automatically removing orphaned layers. The storage driver also prevents the creation of multiple inodes per file, effectively avoiding inode exhaustion in build scenarios and at-scale.
LCFS operates directly on top of block devices, as opposed to two filesystems that are then merged. The filesystem also directly manages at the container image’s layer level, effectively eliminating the overhead of having a second filesystem that is later merged.
“Our immediate plan is to improve the experience with stateless and stateful containers, but this is just a start,” said Gou Rao, CTO of Portworx. “The future direction is to enhance LCFS with cluster-level operations, richer container statistics, and pave the way toward content integrity of container images.”
Running containers on the same server is often limited by the side effects that come from mapping container behavior over general filesystems. The approach impacts the entire lifecycle: building, launching, reading data, and exiting containers.
While file systems were built with the expectation that content is read/writeable, Docker images are constructed using many read-only layers and a single read-write layer. As more containers are launched off of the same image, reading a file within a container requires traversing all the other containers running that image.
Key advantages:
- Layers are managed directly: LCFS inherently understands layers, their different states, and is able to directly track and manage layers
- Clone independence: The filesystem creates and runs container images as independent entities, from an underlying filesystem perspective. Each new instantiation of the same Docker image is an independent clone, at the read-only layer
- Containers in clusters: It is also very well-suited for clustered operation, optimizes for common data patterns and avoids inheriting behavior that overlaps
Unlike non-filesystem storage drivers, which create multiple copies of the same image layers in memory, LCFS doesn’t cache pages, enabling maximized host memory for containers. For added space management, LCFS improves garbage collection by automatically removing orphaned layers. The storage driver also prevents the creation of multiple inodes per file, effectively avoiding inode exhaustion in build scenarios and at-scale.
Become a subscriber of App Developer Magazine for just $5.99 a month and take advantage of all these perks.
MEMBERS GET ACCESS TO
- - Exclusive content from leaders in the industry
- - Q&A articles from industry leaders
- - Tips and tricks from the most successful developers weekly
- - Monthly issues, including all 90+ back-issues since 2012
- - Event discounts and early-bird signups
- - Gain insight from top achievers in the app store
- - Learn what tools to use, what SDK's to use, and more
Subscribe here
