Freestyle Docs

Freestyle / Guides

How to Use S3 as a Git Remote in a Sandbox

Clone and push Git repositories stored in an S3 bucket, keep large files in Git LFS in the same bucket, and scope storage credentials to one repository prefix at the edge.

Store Git repositories in an S3 bucket and clone, fetch, and push them from a sandbox. There is no Git server: the bucket is the remote, and each repository is a key prefix such as s3://my-bucket/repos/<repoId>/. The sandbox never holds the storage key. A TLS rule with an s3 transform signs the sandbox’s requests at the edge and returns 403 for any request outside that repository’s prefix.

This guide uses git-remote-s3, a Git remote helper for s3://bucket/prefix URLs. It stores each branch as one Git bundle under the prefix. The same package ships git-lfs-s3, a Git LFS transfer agent that stores large files under <prefix>/lfs/. Any S3-compatible store works with a different endpoint.

Prepare the Controller

Run the TypeScript examples in your trusted controller, outside the sandbox. Install the SDK there:

pnpm add freestyle@latest
bun add freestyle@latest
npm install freestyle@latest
yarn add freestyle@latest

Supply these environment variables to the controller:

  • FREESTYLE_API_KEY: your Freestyle API key.
  • AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY: a key with access to the bucket.

Install git-remote-s3

Boot an Ubuntu sandbox with public egress and install Git, Git LFS, and the helpers. Version 0.4.2 is pinned because the next step patches it.

import { Freestyle } from "freestyle";

const freestyle = new Freestyle();

const { vm, vmId } = await freestyle.vms.create({
  snapshotId: "freestyle/ubuntu",
  firewall: { rules: [{ action: "allow", source: {}, destination: { public: true } }] },
});

const install = await vm.exec(
  "sudo apt-get update -qq && sudo apt-get install -y -qq git git-lfs pipx && " +
    "git lfs install --skip-repo && " +
    "pipx install git-remote-s3==0.4.2 && " +
    "sudo ln -sf /home/ubuntu/.local/bin/git-remote-s3 /home/ubuntu/.local/bin/git-lfs-s3 /usr/local/bin/",
);
if (install.statusCode !== 0) throw new Error(install.stderr);

Do this once in a base snapshot to skip the install on every sandbox.

Patch the Helpers for a Prefix-Scoped Rule

git-remote-s3 0.4.2 lists refs under the bare prefix repos/<repoId>, which a rule scoped to repos/<repoId>/ rejects. The patch adds the trailing slash. The script exits with an error if the patch target is missing.

const patch = `py=$(head -1 /home/ubuntu/.local/bin/git-remote-s3 | sed 's/^#!//')
"$py" - <<'EOF'
import pathlib, git_remote_s3
pkg = pathlib.Path(git_remote_s3.__file__).parent

def patch(name, old, new):
    path = pkg / name
    src = path.read_text()
    if old not in src:
        raise SystemExit(f"patch target not found in {name}: {old}")
    path.write_text(src.replace(old, new))

for old in ("Bucket=bucket, Prefix=prefix)", "Bucket=bucket, Prefix=prefix, ContinuationToken"):
    patch("remote.py", old, old.replace("Prefix=prefix", 'Prefix=prefix.rstrip("/") + "/"'))
EOF`;

const patched = await vm.exec(patch);
if (patched.statusCode !== 0) throw new Error(patched.stderr);

Grant the Sandbox One Repository

Create a TLS rule for the sandbox, scoped to one repository prefix. End the prefix with /: repos/p1 also matches repos/p10.

const endpoint = "https://s3.us-west-2.amazonaws.com";
const repoId = "my-repo";

const READ = ["GetObject", "HeadObject", "ListObjects", "ListObjectsV2"];
const WRITE = [
  ...READ, "PutObject", "DeleteObject",
  "CreateMultipartUpload", "UploadPart", "ListParts",
  "CompleteMultipartUpload", "AbortMultipartUpload", "ListMultipartUploads",
];

const { id: ruleId } = await freestyle.tls.rules.create({
  action: "allow",
  domain: new URL(endpoint).host,
  source: { vmId },
  destination: { public: true },
  transform: [{
    s3: {
      region: "us-west-2",
      credentials: {
        accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
        secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
      },
      scope: {
        bucket: "my-bucket",
        prefix: `repos/${repoId}/`,
        operations: WRITE,
      },
    },
  }],
});

READ is enough for clone, fetch, and LFS downloads. Push needs WRITE. git-lfs-s3 uploads files over 8 MiB in parts, which uses the multipart operations.

Point the Sandbox at the Edge

Write an AWS profile with the endpoint, path-style addressing, the system CA bundle, and placeholder keys. The edge replaces the placeholder signature with one made from the rule’s credentials.

await vm.exec("mkdir -p /home/ubuntu/.aws");
await vm.fs.writeTextFile(
  "/home/ubuntu/.aws/config",
  `[default]
region = us-west-2
endpoint_url = ${endpoint}
ca_bundle = /etc/ssl/certs/ca-certificates.crt
request_checksum_calculation = when_required
s3 =
  addressing_style = path
`,
);
await vm.fs.writeTextFile(
  "/home/ubuntu/.aws/credentials",
  `[default]
aws_access_key_id = placeholder
aws_secret_access_key = placeholder
`,
);
  • ca_bundle points boto3 at the system trust store, which holds the Freestyle platform CA.
  • request_checksum_calculation = when_required stops boto3 from adding checksum trailers to uploads, which the edge rejects.

Push, Clone, and Fetch

Run these commands inside the sandbox. Create a repository and push it:

git init -b main repo && cd repo
git remote add origin s3://my-bucket/repos/my-repo
echo hello > README.md
git add README.md && git commit -m "Start"
git push origin HEAD:refs/heads/main

Clone it into another directory, or another sandbox with its own rule:

git clone s3://my-bucket/repos/my-repo repo-copy
git -C repo-copy fetch origin

A request for another repository returns 403:

git ls-remote s3://my-bucket/repos/other-repo

Pushes behave differently from a Git server:

  • Push named refs only, in the form HEAD:refs/heads/<branch>. To push a detached commit, create a branch for it first.
  • A push rejected with stale remote means another writer updated the branch. Fetch, merge, and push again.

Store Large Files With Git LFS

Track only the large binary files, by pattern. Everything else stays in the Git bundle. git-lfs-s3 install sets the LFS transfer agent for the remote in the repository’s local config.

cd repo
git-lfs-s3 install --remote origin
git lfs track "*.psd" "*.mp4" "*.zip"
git add .gitattributes assets/
git commit -m "Add assets"
git push origin HEAD:refs/heads/main

LFS objects are stored at repos/my-repo/lfs/<sha256>, inside the rule’s scope.

A clone has no LFS config until git-lfs-s3 install runs in it. Clone with smudge off, install the agent, then download the LFS files:

GIT_LFS_SKIP_SMUDGE=1 git clone s3://my-bucket/repos/my-repo repo-copy
cd repo-copy
git-lfs-s3 install --remote origin
git lfs pull origin

Remove Access

Delete the rule from the controller:

await freestyle.tls.rules.delete(ruleId);

Rules are also deleted with their sandbox. The repository stays in the bucket.

esc