New SDK Creates Infinite File Systems for AI Agents
A new JavaScript/TypeScript SDK called Archil has been released for creating infinite file systems for AI agents that sync directly to S3. The tool is aimed at product engineers building out the infrastructure for more complex, stateful AI agent applications.
Archil was founded by Hunter Leath, a cloud storage veteran who previously led Amazon's Elastic File System (EFS) product and Netflix's cloud storage team. The company, formerly known as Regatta Storage, is backed by Y Combinator and has raised $6.7M to address the storage bottlenecks faced by data-intensive applications. The core problem Archil addresses is that object storage like Amazon S3 is not a file system and lacks features like atomic renames and file locking needed for many applications. Existing tools that mount S3 as a file system, such as s3fs, often suffer from performance and consistency issues, while block storage like EBS is expensive and has limited, non-shareable capacity. Archil functions as a high-speed, durable caching layer between servers and object storage. When an Archil disk is mounted, it connects to a managed fleet of instances with NVMe devices that provide read-through and write-back caching, offering sub-millisecond latency for frequently accessed data. This architecture allows it to provide a fully POSIX-compliant file system interface. For stateful AI agents, this "infinite filesystem" is critical. It allows agents to maintain state, manage large datasets for training or Retrieval-Augmented Generation (RAG), and access model artifacts without being constrained by local disk capacity. This approach helps make applications stateless, simplifying data management as agents don't need complex logic for checkpointing or handling cold-starts. The system uses a custom storage protocol designed for high performance, especially with workloads involving many small files, a common pain point with other S3-based file systems. While specific benchmarks aren't published, Archil claims its high-speed data layer provides 30x lower latency than accessing S3 directly. Unlike some alternatives, Archil writes data back to the user's S3 bucket in its native object format. This means the data remains accessible directly via the S3 API for other services and workflows, avoiding vendor lock-in where data is stored in a proprietary format.
