Freestyle Docs

Freestyle / Guides

How to Run GNOME in a Sandbox

Build a GNOME desktop with TigerVNC, Chromium, and noVNC, then snapshot and reuse it.

This guide builds a complete GNOME desktop in a VM. GNOME Shell provides the desktop and window manager, TigerVNC owns the virtual X display, and noVNC puts the session in a browser. Chromium runs as a normal desktop application.

The tested stack is GNOME Shell 48 on Debian 13, at 1440x900 on X display :1. TigerVNC listens only on loopback port 5901; noVNC serves the browser client on port 6080.

Why GNOME Uses an Xorg Session

GNOME normally starts through a display manager and prefers Wayland. A TigerVNC-owned desktop instead needs GNOME’s Xorg session, so Xtigervnc can be both the X server and VNC server.

GNOME Shell also expects graphics acceleration. This VM has no physical display or GPU, so the launcher sets LIBGL_ALWAYS_SOFTWARE=1 and lets Mesa render the desktop in software.

Use this full desktop when a person needs GNOME applications, the Activities overview, or a general remote workstation. For one browser controlled by an agent, use the smaller Chrome computer-use stack.

Install the SDK

pnpm add freestyle
bun add freestyle
npm install freestyle
yarn add freestyle
 
export FREESTYLE_API_KEY="your-api-key"

Install GNOME, TigerVNC, and Chromium

GNOME has a large package graph. Run the install in a background systemd unit so it is not tied to one buffered vm.exec() request:

import { freestyle } from "freestyle";

async function runSetup(vm, name: string, script: string) {
  const scriptPath = `/root/setup-${name}.sh`;
  const donePath = `/tmp/setup-${name}.done`;
  const failedPath = `/tmp/setup-${name}.failed`;
  const unit = `setup-${name}`;

  await vm.fs.writeTextFile(
    scriptPath,
    `#!/bin/bash
set -euxo pipefail
export DEBIAN_FRONTEND=noninteractive
rm -f ${donePath} ${failedPath}
trap 'touch ${failedPath}' ERR
${script}
touch ${donePath}
`,
  );

  await vm.exec(
    `chmod +x ${scriptPath} && systemd-run --unit=${unit} --collect ${scriptPath}`,
  );

  for (let i = 0; i < 240; i++) {
    const probe = await vm.exec(
      `if test -f ${donePath}; then echo DONE; ` +
        `elif test -f ${failedPath}; then echo FAILED; ` +
        `journalctl -u ${unit} --no-pager -n 120; ` +
        `else systemctl show ${unit} -p ActiveState -p Result --no-pager; fi`,
    );
    const output = `${probe.stdout ?? ""}${probe.stderr ?? ""}`;
    if (output.includes("DONE")) return;
    if (output.includes("FAILED")) throw new Error(output);
    await new Promise((resolve) => setTimeout(resolve, 5_000));
  }

  throw new Error(`${unit} did not finish`);
}

const { vm: builder } = await freestyle.vms.create({
  slug: "gnome-tigervnc-builder",
  idleTimeoutSeconds: null,
});

await runSetup(
  builder,
  "gnome",
  `apt-get update -qq
apt-get install -y -qq --no-install-recommends \
  ca-certificates curl dbus-x11 \
  gnome-session gnome-session-xsession gnome-shell \
  gnome-terminal nautilus \
  tigervnc-standalone-server tigervnc-common tigervnc-tools \
  novnc websockify x11-utils xdotool scrot \
  chromium fonts-liberation fonts-cantarell sudo`,
);

await builder.exec(
  "id desktop >/dev/null 2>&1 || " +
    "useradd --create-home --shell /bin/bash desktop",
);

await builder.fs.writeTextFile(
  "/etc/sudoers.d/desktop",
  "desktop ALL=(ALL:ALL) NOPASSWD: ALL\n",
);
await builder.exec(
  "chmod 0440 /etc/sudoers.d/desktop && " +
    "visudo -cf /etc/sudoers.d/desktop",
);

await builder.fs.writeTextFile(
  "/usr/share/gnome-session/sessions/gnome-vnc.session",
  `[GNOME Session]
Name=GNOME VNC
RequiredComponents=org.gnome.Shell;org.gnome.SettingsDaemon.A11ySettings;org.gnome.SettingsDaemon.Color;org.gnome.SettingsDaemon.Datetime;org.gnome.SettingsDaemon.Housekeeping;org.gnome.SettingsDaemon.Keyboard;org.gnome.SettingsDaemon.MediaKeys;org.gnome.SettingsDaemon.PrintNotifications;org.gnome.SettingsDaemon.Rfkill;org.gnome.SettingsDaemon.ScreensaverProxy;org.gnome.SettingsDaemon.Sharing;org.gnome.SettingsDaemon.Smartcard;org.gnome.SettingsDaemon.Sound;org.gnome.SettingsDaemon.UsbProtection;org.gnome.SettingsDaemon.Wacom;org.gnome.SettingsDaemon.XSettings;
`,
);

await builder.exec("systemctl mask --now upower.service || true");

useradd creates desktop with a locked password. GNOME can still start the account through systemd, but a graphical sudo prompt cannot authenticate it. The validated sudoers rule deliberately makes sudo non-interactive so the desktop is immediately useful in a disposable, single-user sandbox.

This also means anyone controlling the desktop can become root. For a longer-lived or shared VM, use the runtime password option below instead. The password is applied after restoring the snapshot, so it is not preserved in every VM created from that snapshot.

The important dependencies are:

The custom gnome-vnc session omits only GNOME Settings Daemon’s power plugin. The standard session treats that plugin as required, but UPower is not useful on this headless VM and can fail while waiting for PolicyKit. Without the custom session, GNOME may show its fatal error screen a few minutes after appearing to start successfully.

Start GNOME on TigerVNC

Run the desktop as an unprivileged desktop user. The launcher starts Xtigervnc, waits for X, then starts GNOME in a D-Bus session:

await builder.fs.writeTextFile(
  "/usr/local/bin/start-gnome-tigervnc",
  `#!/bin/bash
set -euo pipefail

export USER=desktop
export HOME=/home/desktop
export DISPLAY=:1
export XDG_RUNTIME_DIR=/run/desktop-gnome
export XDG_SESSION_TYPE=x11
export XDG_CURRENT_DESKTOP=GNOME
export XDG_SESSION_DESKTOP=gnome
export GDMSESSION=gnome
export LIBGL_ALWAYS_SOFTWARE=1

mkdir -p "$XDG_RUNTIME_DIR" "$HOME/.vnc"
chmod 700 "$XDG_RUNTIME_DIR"
rm -f /tmp/.X1-lock /tmp/.X11-unix/X1

vnc_pid=
desktop_pid=

/usr/bin/Xtigervnc :1 \
  -geometry 1440x900 \
  -depth 24 \
  -rfbport 5901 \
  -localhost \
  -SecurityTypes None \
  -AlwaysShared=1 &
vnc_pid=$!

cleanup() {
  if [ -n "\${desktop_pid:-}" ]; then
    kill "$desktop_pid" 2>/dev/null || true
  fi
  if [ -n "\${vnc_pid:-}" ]; then
    kill "$vnc_pid" 2>/dev/null || true
  fi
}
trap cleanup EXIT INT TERM

for _ in $(seq 1 30); do
  xdpyinfo >/dev/null 2>&1 && break
  sleep 1
done
xdpyinfo >/dev/null

dbus-launch --exit-with-session gnome-session --session=gnome-vnc &
desktop_pid=$!

while kill -0 "$vnc_pid" 2>/dev/null &&
      kill -0 "$desktop_pid" 2>/dev/null; do
  sleep 2
done

exit 1
`,
);

-localhost keeps raw VNC off the VM network. -SecurityTypes None is safe only because websockify is the sole VNC client and connects over loopback.

TigerVNC may log DRI3 or render-node warnings. They are expected on the software-rendered display and do not prevent GNOME Shell from running.

Add the Desktop Services

The GNOME unit owns the display and session. noVNC bridges browser WebSockets to TigerVNC. Chromium has its own unit so it can restart independently of the desktop.

await builder.fs.writeTextFile(
  "/etc/systemd/system/tigervnc-gnome.service",
  `[Unit]
Description=GNOME desktop on TigerVNC
After=network.target

[Service]
Type=simple
User=desktop
Environment=HOME=/home/desktop
RuntimeDirectory=desktop-gnome
RuntimeDirectoryMode=0700
ExecStart=/usr/local/bin/start-gnome-tigervnc
Restart=always
RestartSec=3
KillMode=control-group
TimeoutStopSec=10

[Install]
WantedBy=multi-user.target
`,
);

await builder.fs.writeTextFile(
  "/etc/systemd/system/novnc.service",
  `[Unit]
Description=noVNC web client
After=tigervnc-gnome.service
Requires=tigervnc-gnome.service

[Service]
ExecStart=/usr/bin/websockify --web /usr/share/novnc 6080 localhost:5901
Restart=always

[Install]
WantedBy=multi-user.target
`,
);

await builder.fs.writeTextFile(
  "/etc/systemd/system/chromium-gnome.service",
  `[Unit]
Description=Chromium in the GNOME desktop
After=tigervnc-gnome.service
Requires=tigervnc-gnome.service

[Service]
User=desktop
Environment=DISPLAY=:1
Environment=HOME=/home/desktop
Environment=XDG_RUNTIME_DIR=/run/desktop-gnome
ExecStartPre=/bin/bash -lc 'for i in {1..60}; do xdpyinfo >/dev/null 2>&1 && exit 0; sleep 1; done; exit 1'
ExecStartPre=/bin/bash -lc 'mkdir -p "$HOME/.config/chromium-gnome"; rm -f "$HOME/.config/chromium-gnome"/Singleton*'
ExecStart=/usr/bin/chromium --no-sandbox --disable-dev-shm-usage --no-first-run --disable-default-apps --user-data-dir=/home/desktop/.config/chromium-gnome --start-maximized https://example.com
Restart=always
RestartSec=2
TimeoutStopSec=10

[Install]
WantedBy=multi-user.target
`,
);

await builder.exec(
  "chmod +x /usr/local/bin/start-gnome-tigervnc && " +
    "systemctl daemon-reload && " +
    "systemctl enable --now tigervnc-gnome novnc chromium-gnome",
);

The dedicated Chromium profile avoids stale singleton locks after a snapshot restore. For Google Chrome Stable, install it with Run Chromium and Chrome and replace the Chromium executable and profile path.

Verify and Snapshot GNOME

Wait for the actual desktop, browser window, and noVNC page before snapshotting:

const ready = await builder.exec(
  `for i in $(seq 1 150); do
  curl -fsS http://127.0.0.1:6080/vnc.html >/dev/null &&
  DISPLAY=:1 xdpyinfo >/dev/null 2>&1 &&
  pgrep -x gnome-session-b >/dev/null &&
  pgrep -x gnome-shell >/dev/null &&
  systemctl is-active --quiet chromium-gnome &&
  DISPLAY=:1 xwininfo -root -tree | grep -q Chromium &&
  pgrep -f 'Xtigervnc :1' >/dev/null &&
  exit 0
  sleep 2
done
journalctl -u tigervnc-gnome -u novnc -u chromium-gnome --no-pager -n 240
exit 1`,
);

if (ready.statusCode !== 0) {
  throw new Error(`${ready.stdout}\n${ready.stderr}`);
}

await new Promise((resolve) => setTimeout(resolve, 150_000));

const stable = await builder.exec(
  `systemctl is-active --quiet tigervnc-gnome novnc chromium-gnome &&
pgrep -x gnome-shell >/dev/null &&
! pgrep -x gnome-session-failed >/dev/null`,
);

if (stable.statusCode !== 0) {
  throw new Error(`${stable.stdout}\n${stable.stderr}`);
}

const windows = await builder.exec(
  "DISPLAY=:1 xwininfo -root -tree | grep -E 'GNOME Shell|Chromium'",
);
console.log(windows.stdout);

const { snapshotId } = await builder.snapshot();
await builder.delete();

The snapshot captures the running GNOME session. A VM created from it resumes GNOME Shell, Chromium, TigerVNC, and noVNC together.

Open the GNOME Desktop

Create a VM from the snapshot and map a domain to noVNC’s port:

const { vm, vmId } = await freestyle.vms.create({
  slug: "gnome-desktop",
  snapshotId,
  idleTimeoutSeconds: null,
});

Optional: Require a sudo Password

The snapshot defaults to passwordless sudo. To require a password for this VM, set it before exposing noVNC, then replace the NOPASSWD rule with a normal sudoers rule:

const desktopPassword = process.env.DESKTOP_SUDO_PASSWORD;
if (!desktopPassword || /[\r\n]/.test(desktopPassword)) {
  throw new Error("Set DESKTOP_SUDO_PASSWORD without newline characters");
}

const passwordFile = "/root/desktop-password";
await vm.fs.writeTextFile(
  passwordFile,
  `desktop:${desktopPassword}\n`,
);

const configured = await vm.exec(
  `set -eu
trap 'rm -f ${passwordFile}' EXIT
chmod 0600 ${passwordFile}
chpasswd < ${passwordFile}
printf '%s\n' 'desktop ALL=(ALL:ALL) ALL' > /etc/sudoers.d/desktop
chmod 0440 /etc/sudoers.d/desktop
visudo -cf /etc/sudoers.d/desktop`,
);

if (configured.statusCode !== 0) {
  throw new Error(`${configured.stdout}\n${configured.stderr}`);
}

The temporary file is removed before the API call returns. The password applies only to sudo inside this VM; it does not add a GNOME login screen or protect the noVNC URL.

Optional: Require a TigerVNC Password in the Web Viewer

TigerVNC provides the VNC server, while noVNC is the browser client. The default launcher uses SecurityTypes None, so noVNC connects without a VNC password. To make the browser prompt for one, create a TigerVNC password file and switch the server to VncAuth after restoring the snapshot:

const vncPassword = process.env.VNC_PASSWORD;
if (!vncPassword || !/^[\x21-\x7e]{6,8}$/.test(vncPassword)) {
  throw new Error("Set VNC_PASSWORD to 6-8 printable ASCII characters");
}

const passwordInput = "/root/vnc-password";
const vncPasswordFile = "/home/desktop/.vnc/passwd";
const launcherPath = "/usr/local/bin/start-gnome-tigervnc";

await vm.fs.writeTextFile(passwordInput, `${vncPassword}\n`);

const passwordResult = await vm.exec(
  `set -eu
trap 'rm -f ${passwordInput}' EXIT
chmod 0600 ${passwordInput}
install -d -m 0700 -o desktop -g desktop /home/desktop/.vnc
/usr/bin/tigervncpasswd -f < ${passwordInput} > ${vncPasswordFile}
chown desktop:desktop ${vncPasswordFile}
chmod 0600 ${vncPasswordFile}`,
);

if (passwordResult.statusCode !== 0) {
  throw new Error(`${passwordResult.stdout}\n${passwordResult.stderr}`);
}

const launcher = await vm.fs.readTextFile(launcherPath);
if (!launcher.includes("-SecurityTypes None")) {
  throw new Error("TigerVNC launcher is already authenticated or has changed");
}

await vm.fs.writeTextFile(
  launcherPath,
  launcher.replace(
    "-SecurityTypes None",
    `-SecurityTypes VncAuth -PasswordFile "$HOME/.vnc/passwd"`,
  ),
);

const restarted = await vm.exec(
  `chmod +x ${launcherPath}
systemctl restart tigervnc-gnome
systemctl restart novnc chromium-gnome
for i in $(seq 1 90); do
  systemctl is-active --quiet tigervnc-gnome novnc chromium-gnome &&
  DISPLAY=:1 xdpyinfo >/dev/null 2>&1 &&
  pgrep -x gnome-shell >/dev/null &&
  exit 0
  sleep 1
done
exit 1`,
);

if (restarted.statusCode !== 0) {
  throw new Error(`${restarted.stdout}\n${restarted.stderr}`);
}

TigerVNC accepts passwords of at least six characters, but only the first eight characters are significant, so this example requires a 6-to-8-character value. Set the password on each runtime VM instead of storing it in the snapshot.

The same noVNC URL now opens a password prompt before showing GNOME. There is no username; enter VNC_PASSWORD.

Turn Off the TigerVNC Password

To return a runtime VM to the original passwordless configuration, restore SecurityTypes None, remove the password file, and restart the desktop:

const launcherPath = "/usr/local/bin/start-gnome-tigervnc";
const authenticatedFlags =
  `-SecurityTypes VncAuth -PasswordFile "$HOME/.vnc/passwd"`;

const launcher = await vm.fs.readTextFile(launcherPath);
if (!launcher.includes(authenticatedFlags)) {
  throw new Error("TigerVNC password authentication is not enabled");
}

await vm.fs.writeTextFile(
  launcherPath,
  launcher.replace(authenticatedFlags, "-SecurityTypes None"),
);

const disabled = await vm.exec(
  `rm -f /home/desktop/.vnc/passwd
chmod +x ${launcherPath}
systemctl restart tigervnc-gnome
systemctl restart novnc chromium-gnome
for i in $(seq 1 90); do
  systemctl is-active --quiet tigervnc-gnome novnc chromium-gnome &&
  DISPLAY=:1 xdpyinfo >/dev/null 2>&1 &&
  pgrep -x gnome-shell >/dev/null &&
  exit 0
  sleep 1
done
exit 1`,
);

if (disabled.statusCode !== 0) {
  throw new Error(`${disabled.stdout}\n${disabled.stderr}`);
}

Restarting TigerVNC disconnects current viewers. Reload the same noVNC URL and it connects without asking for a password.

Now map the domain:

const domain = `gnome-${crypto.randomUUID().slice(0, 8)}.style.dev`;
await freestyle.domains.mappings.create({
  domain,
  vmId,
  vmPort: 6080,
});

const desktopUrl =
  `https://${domain}/vnc.html?` +
  "autoconnect=true&reconnect=true&reconnect_delay=2000&" +
  "resize=scale&path=websockify";

console.log(desktopUrl);

The browser should show GNOME’s top bar and Activities overview with Chromium open. Mouse and keyboard input travel through noVNC to TigerVNC.

Confirm the restored VM from the API:

const state = await vm.exec(
  `systemctl is-active tigervnc-gnome novnc chromium-gnome
pgrep -a gnome-shell
DISPLAY=:1 xwininfo -root -tree | grep Chromium`,
);
console.log(state.stdout);

Give a Computer-Use Agent the GNOME Desktop

A computer-use agent needs an observation and a small set of actions. scrot captures the entire GNOME desktop as the next visual observation. xdotool moves and clicks the pointer, types text, and sends key combinations. The agent can therefore operate Chromium, Terminal, Files, native dialogs, and other GNOME applications through the same screenshot-and-coordinate loop.

noVNC remains the human channel for watching or taking over. The agent does not need to connect to VNC or run an automation daemon inside the VM. This example exposes the computer-use actions from a stateless function: each invocation attaches to the VM, performs one action through vm.exec(), and returns. The VM keeps the stateful GNOME session, open applications, cookies, and windows:

import { Freestyle } from "freestyle";

type Env = {
  FREESTYLE_API_KEY: string;
  GNOME_VM_ID: string;
  COMPUTER_USE_TOKEN: string;
};

type ComputerAction =
  | { type: "screenshot" }
  | { type: "click"; x: number; y: number; button?: number }
  | { type: "move"; x: number; y: number }
  | { type: "type"; text: string }
  | { type: "key"; key: string };

const width = 1440;
const height = 900;

function coordinate(value: unknown, limit: number) {
  if (!Number.isInteger(value) || (value as number) < 0 || (value as number) >= limit) {
    throw new Error("Coordinate is outside the desktop");
  }
  return value as number;
}

export async function handleComputerUse(request: Request, env: Env) {
  if (
    request.headers.get("authorization") !==
    `Bearer ${env.COMPUTER_USE_TOKEN}`
  ) {
    return new Response("Unauthorized", { status: 401 });
  }

  const action = (await request.json()) as ComputerAction;
  const freestyle = new Freestyle({ apiKey: env.FREESTYLE_API_KEY });
  const { vm } = await freestyle.vms.get({ vmId: env.GNOME_VM_ID });

  async function desktop(command: string) {
    const result = await vm.exec(
      `runuser -u desktop -- env HOME=/home/desktop DISPLAY=:1 ${command}`,
    );
    if (result.statusCode !== 0) {
      throw new Error(`${result.stdout ?? ""}\n${result.stderr ?? ""}`);
    }
  }

  if (action.type === "screenshot") {
    const path = `/tmp/computer-use-${crypto.randomUUID()}.png`;
    try {
      await desktop(`scrot --overwrite ${path}`);
      const png = await vm.fs.readFile(path);
      return new Response(png, { headers: { "content-type": "image/png" } });
    } finally {
      await vm.fs.remove(path).catch(() => {});
    }
  }

  if (action.type === "click" || action.type === "move") {
    const x = coordinate(action.x, width);
    const y = coordinate(action.y, height);

    if (action.type === "move") {
      await desktop(`xdotool mousemove --sync ${x} ${y}`);
    } else {
      const button = action.button ?? 1;
      if (!Number.isInteger(button) || button < 1 || button > 5) {
        throw new Error("Mouse button must be between 1 and 5");
      }
      await desktop(`xdotool mousemove --sync ${x} ${y} click ${button}`);
    }
  } else if (action.type === "type") {
    const path = `/tmp/computer-use-${crypto.randomUUID()}.txt`;
    try {
      await vm.fs.writeTextFile(path, action.text);
      await desktop(`xdotool type --clearmodifiers --delay 12 --file ${path}`);
    } finally {
      await vm.fs.remove(path).catch(() => {});
    }
  } else if (action.type === "key") {
    if (!/^[A-Za-z0-9_+:-]+$/.test(action.key)) {
      throw new Error("Unsupported key expression");
    }
    await desktop(`xdotool key --clearmodifiers ${action.key}`);
  } else {
    throw new Error("Unsupported computer-use action");
  }

  return Response.json({ ok: true });
}

Give the agent five tools matching the ComputerAction variants: observe the screen, click, move, type, and press a key. After each action, request another screenshot and pass it back to the model. Keep coordinates in the same 1440x900 space used by TigerVNC so the model’s selected point maps directly to the desktop.

Keep FREESTYLE_API_KEY and COMPUTER_USE_TOKEN in the function’s secret store. The bearer token protects this narrow computer-use API; the Freestyle API key remains capable of managing the entire VM.

Use this whole-desktop path when the agent must cross application boundaries. For browser-only agents that benefit from DOM inspection and selectors, use the Chrome and Playwright computer-use guide.

Add to your Agent

Open Markdown
esc