Run real microVMs on your Mac

On an Apple Silicon Mac you can run real Firecracker microVMs locally: a Lima Linux guest with nested virtualization provides /dev/kvm, and the runtime-manager runs inside it with RUNTIME_BACKEND=firecracker.

Requirements

• Apple Silicon M3 or newer and macOS 15 or newer. Nested virtualization in Virtualization.framework needs both.
• Lima (brew install lima).

The Lima template (infra/lima/firecracker-dev.yaml) uses vmType: vz with nestedVirtualization: true, 4 CPUs, 8 GiB RAM and a 30 GiB disk, and installs the pinned Firecracker v1.15.1 release plus the jailer. Provisioning fails with a /dev/kvm missing error on an M1/M2 or a pre-macOS-15 host. There, use the KVM CI workflow (.github/workflows/microvm-integration.yml), a remote Linux host, or the Docker backend (RUNTIME_BACKEND=docker), which stays the zero-setup macOS path.

What you get

Without LANTERN_SCHEDULER_GRPC_ADDR the control-plane uses a stub scheduler client, and without LANTERN_DEFAULT_MANAGER_ADDR the scheduler keeps a log-only dialer. Both are fine for dashboard work and are rejected in production.

Setup

From the repo on the Mac (it must live under your home directory, which the VM mounts read-only at the same path):

brew install lima
make lima-up           # create/start "firecracker-dev"; verify /dev/kvm, firecracker, jailer; install guest build deps
make lima-e2e          # build harness + manager + image in the VM, run the jailed lifecycle e2e, print the timing table
make lima-broker-e2e   # broker egress mode (ADR 0026): no NIC, vsock-only, credential injection

Both e2e targets fail if a run leaves a firecracker process, snapshot directory or jail behind. If another run already holds the manager port (:50054, broker :51054) they wait for it, up to LIMA_PORT_WAIT_SECS (default 1800), instead of killing it. Builds refuse to start when the VM disk is over LIMA_DISK_MAX_PCT (default 85%); make lima-clean frees the caches. All build output goes to ~/.cache/lantern-lima in the VM, never into the repo.

Manual steps

What the targets automate, for debugging. Run them yourself only when a target fails and you need to see why.

Start the Lima guest

limactl start infra/lima/firecracker-dev.yaml   # creates "firecracker-dev"
limactl shell firecracker-dev                   # /dev/kvm exists in here

Smoke-boot a microVM

Inside the guest, with the official Firecracker CI artifacts (v1.15.1 kernel series):

mkdir -p ~/fc-demo && cd ~/fc-demo
curl -sO http://spec.ccfc.min.s3.amazonaws.com/firecracker-ci/v1.15/aarch64/vmlinux-5.10.245
curl -sO http://spec.ccfc.min.s3.amazonaws.com/firecracker-ci/v1.15/aarch64/ubuntu-22.04.ext4
cat > vm.json <<'EOF'
{
  "boot-source": {"kernel_image_path": "vmlinux-5.10.245", "boot_args": "keep_bootcon console=ttyS0 reboot=k panic=1 pci=off"},
  "drives": [{"drive_id": "rootfs", "path_on_host": "ubuntu-22.04.ext4", "is_root_device": true, "is_read_only": false}],
  "machine-config": {"vcpu_count": 2, "mem_size_mib": 512}
}
EOF
firecracker --no-api --config-file vm.json   # boots to a root login on ttyS0

On an M4 Max this boots Ubuntu 22.04 to login in about 1.6 s.

Run the lifecycle test

The repo is mounted read-only at the same host path inside the guest. The test boots a VM, then snapshots, stops and restores it, forks three copies and prints timings. Inside the guest:

cd /path/to/repo

# Build deps (first time only)
sudo apt-get install -y musl-tools e2fsprogs jq curl protobuf-compiler
rustup target add aarch64-unknown-linux-musl

# The repo is read-only in the guest: build into a cache under ~
TARGET=~/.cache/lantern-lima/target

# Harness (musl, static)
cargo build --release --bin lantern-harness --target aarch64-unknown-linux-musl \
  --manifest-path services/harness/Cargo.toml --target-dir "$TARGET"
HARNESS_BIN="$TARGET/aarch64-unknown-linux-musl/release/lantern-harness"

# Kernel + rootfs (sudo for loop-mount; harness passed via env)
sudo FC_HARNESS_BIN="$HARNESS_BIN" \
  bash infra/firecracker/build-image.sh /tmp/fc-image

# Jail root for the fork step
sudo mkdir -p /srv/jailer

# Runtime-manager
cargo build --release --manifest-path services/runtime-manager/Cargo.toml --target-dir "$TARGET"
MANAGER_BIN="$TARGET/release/lantern-runtime-manager"

# The test (root: TAP + firecracker + loop devices)
sudo \
  FC_KERNEL_PATH=/tmp/fc-image/vmlinux \
  FC_ROOTFS_PATH=/tmp/fc-image/rootfs.ext4 \
  RUNTIME_BACKEND=firecracker \
  LANTERN_RUNTIME_BACKEND=firecracker \
  MANAGER_BIN="$MANAGER_BIN" \
  EXPECT_VENDSECRET=1 \
  FIRECRACKER_JAILER_PATH=/usr/local/bin/jailer \
  FIRECRACKER_JAILER_UID=123 \
  FIRECRACKER_JAILER_GID=100 \
  FIRECRACKER_CHROOT_BASE=/srv/jailer \
  LIFECYCLE_TIMING_DIR=/tmp/fc-timing \
  bash infra/firecracker/integration-test.sh

cat /tmp/fc-timing/lifecycle-timing.md

The fork step runs only when FIRECRACKER_JAILER_PATH points at the jailer (the Lima provision script installs it at /usr/local/bin/jailer; being on PATH is not enough, because the manager must be started with it). Without it the test prints [integration] SKIP: fork test SKIPPED and still passes.

Clean up

limactl stop firecracker-dev && limactl delete firecracker-dev

Point Lantern at it

Build and run the runtime-manager inside the guest (it needs /dev/kvm) with RUNTIME_BACKEND=firecracker; it listens on 0.0.0.0:50054 by default (LISTEN_ADDR). Then, on the Mac, point the control-plane and scheduler at it through the Lima port forward. These are the variables the stack already reads; the values assume the manager is reachable at localhost:50054:

The repo does not spell out the forward end to end: firecracker-dev.yaml defines no explicit portForwards and infra/lima/README.md only says to use "the Lima port forward". Confirm that localhost:50054 reaches the manager before debugging anything else.

Dev setups without a shared service token warn and allow unauthenticated gRPC; production requires LANTERN_GRPC_SERVICE_TOKEN on every hop. See Isolation tiers for the other backends.

Verify it works

• In the guest, /dev/kvm exists and firecracker --version prints v1.15.1.
• The smoke-boot reaches a root login on ttyS0.
• make lima-e2e ends with ALL ASSERTIONS PASSED, including the fork step, and prints the timing table.
• With the stack pointed at the guest, a scheduled VM's node is not node-stub and GET /v1/runtime/vms/{id} shows it running.

Troubleshooting

Limits

• Nested virtualization is for development. Lima timings are indicative, not bare-metal numbers.
• The examples are aarch64 (Apple Silicon) only.
• The repo mount is read-only, so build outputs land inside the guest.
• Not a supported production topology.

Related: Sandbox lifecycle, Egress and credentials.