AWS Lambda functions can now mount Amazon S3 buckets as file systems with S3 Files - AWS

Open original

AWS Lambda now supports S3 Files, letting functions mount S3 buckets as file systems for direct read/write without object downloads. This simplifies stateful and AI/ML workflows by enabling shared workspaces and automatic checkpointing via durable functions.

  • Lambda functions can now mount S3 buckets directly as file systems (S3 Files, built on EFS) instead of downloading/uploading objects in code
  • Multiple Lambda functions can access the same S3 Files mount simultaneously for shared, real-time workspaces without custom locking
  • Combined with durable functions for checkpointing, this avoids the 512 MB /tmp storage limit for stateful, multi-step workflows like AI/ML pipelines
  • Setup works through the Lambda console, CLI, SDKs, CloudFormation, or SAM, with no extra fee beyond normal Lambda and S3 pricing

AWS Lambda now lets you mount an S3 bucket as a file system using a feature called S3 Files. Behind the scenes, it’s built on Amazon EFS, so your Lambda functions see S3 objects as ordinary files and folders. You don’t have to write code to download, upload or sync objects. Multiple functions can connect to the same S3 Files system at once, sharing data in real time without custom locking or coordination. That direct file-system view is a big deal for stateful or multi-step jobs, especially in AI and machine learning. Imagine an orchestrator Lambda that checks out a code repo into the shared S3 Files volume. Then a set of worker Lambdas can run analysis on that code in parallel, writing results back to the same workspace. Durable functions handle the checkpointing of your workflow, while S3 Files handles the data sharing. You avoid hitting the 512 MB temporary storage limit and don’t juggle object downloads or uploads between steps. Setting it up is straightforward. In the Lambda console—or via AWS CLI, AWS SDKs, CloudFormation or SAM—you point your function to an existing S3 bucket and choose a mount path. S3 Files support is available in every AWS Region that offers both Lambda and S3 Files. Pricing follows your usual Lambda and S3 rates—there’s no extra fee for the file-system layer.

Ask about this article