vmix.nix/lib/images/macos/README.md

# macOS images (Tahoe 26)

Pre-installed, Apple-ID-capable macOS VM images built the same way as the
Windows ones: `makeImage` (unattended install) → templates → `.generalize`
(user, hostname, fresh SMBIOS identity). Runs on QEMU/KVM with OpenCore.

```
vmix build --image macos.images.tahoe.basic --generalize username=sagar,password=secret,hostname=MAC
vmix run ./result --macos --vnc :10 --mem 8192
```

Nix: `macos.images.tahoe.{pe,upstream,basic,remote}` and
`<image>.generalize { username; password; hostname; timezone; locale; seed; … }`.

## How it works: the vmix "PE"

Apple's Recovery (`BaseSystem.dmg`, a plain journaled HFS+ volume) with **one
LaunchDaemon added** (`makeRecoveryPE`): at boot it mounts a `VMIX` volume and
runs `run.sh` from it as root, records the exit status and powers off. That is
the whole automation surface — the equivalent of Windows PE + Autounattend:

* **no GUI is driven**: no OCR, no keystrokes, no screen layouts to learn per
  macOS version; the hook is a launchd plist, stable across releases (same idea
  as AutoNBI/Imagr NetBoot images).
* **observable**: the guest prints `VMIX-*` markers to `/dev/console`, which the
  build reads from QEMU's serial log (`boot-args serial=3 -v`). Kernel panics and
  reboots show up there too. Screenshots are still taken for debugging.
* **offline**: no NIC during the install, and the guest blackholes Apple's
  install/verify endpoints so `startosinstall` never waits on the network. The
  only inputs are the pinned `InstallAssistant.pkg` and `BaseSystem.dmg`.
* **everything else happens offline from the PE too**: templates and generalize
  mount the image's Data volume (rw) and System volume (ro) and edit them
  (`dscl -f` for users, `plutil` for preferences) — the installed macOS is
  never booted for customization, so nothing depends on launchd/BTM approval,
  first-boot agents or auto-login inside the guest. One PE boot ≈ 30 s.

### Pipeline

1. `makeRecoveryPE` — BaseSystem.dmg → raw HFS+ image + `ch.vmix.pe` daemon.
2. `makeImage` — QEMU with: OpenCore boot disk (build variant with serial
   console), the PE, the empty target disk, the VMIX volume (`vmix-install.sh`,
   installer app skeleton) and the whole `InstallAssistant.pkg` mapped as a raw
   disk. The guest script erases the target as APFS, unpacks the app and `dd`s
   the pkg into it as `SharedSupport.dmg` (it is a "pkgdmg": xar + koly footer;
   the bare xar member fails with "pkgdmg is missing a footer"), then runs
   `startosinstall`, which reboots itself through the install phases. The
   installed system's first boot ends at the loginwindow: the driver detects the
   bright screen and powers the VM down. OpenCore is then copied into the image's
   own ESP so it boots with plain OVMF.
3. `customizeImage` — boots the PE with the image attached (OpenCore
   `ScanPolicy` restricted to HFS+ on SATA, so only the PE can boot) and runs the
   template script with `$SYS`/`$DATA` mounted. `pe-lib.sh` has the helpers.
4. `templates/generalize.nix` — user (dscl, admin, home from the user template),
   auto-login (`kcpassword`), Setup Assistant suppression, hostname, locale,
   timezone, keyboard type, container resize, fresh SMBIOS via a new OpenCore
   ESP (`serial`/`mlb` from macserial, MAC + UUID from `seed`).

### Recovery source

`recovery.file` in `upstream.json` points at a content-addressed store path for
the verified Tahoe `BaseSystem.dmg` (Apple's CDN load-balances Sequoia/Tahoe
during the rollout, so a plain fetch is non-deterministic). Reproduce it on any
host with `nix store add-path --name macos-tahoe-BaseSystem.dmg BaseSystem.dmg`.
Drop `recovery.file` to fetch from Apple instead (`fetchRecovery` retries until
the pinned hash matches).

## Reliability

Things QEMU does intermittently, and what handles each (all in `vm-driver.py`
and `vmix-install.sh`; every event is logged with a reason):

* `startosinstall` prepare stalls or crawls — the guest kills and retries it on a
  freshly erased target (free-space watchdog + time cap).
* the installer comes back to the PE instead of the install phase — the PE
  counts boots and simply re-runs the install (max 3).
* the installed system hangs at the Apple logo on first boot — a `system_reset`
  is issued only when the screen is dark and frozen **and** disk and serial
  console are idle, so a slow-but-working boot is never interrupted.
* macOS `shutdown -h` halts to a black screen without an ACPI power-off — an
  idle black screen counts as a completed halt.
* a kernel panic (seen on the serial console) resets the VM.
* a wedged run fails at the 4 h timeout instead of hanging.

`tools/soak.sh <flake> macos.images.tahoe.upstream 3` rebuilds an image N
times and tabulates outcome, duration, boots, resets, panics and retries.
Measured 2026-09-09 on the build host (Ryzen 7 7840HS, ZFS), Tahoe 26.6.2,
VirtualSMC-only, PE install — 3 of 3 builds completed:

| run | minutes | kernel boots | prepare tries | panics (self-recovered) | reboot deaths |
|-----|---------|--------------|---------------|-------------------------|---------------|
| 1   | 30      | 8            | 1             | 2                       | 0             |
| 2   | 26      | 7            | 1             | 1                       | 0             |
| 3   | 26      | 7            | 1             | 1                       | 0             |

What still happens: a kernel panic at one of the guest's own reboots during the
install (GPF in launchd's context right after `MACH Reboot`). XNU reboots
itself within seconds and the install continues; the driver only intervenes
if no kernel comes back within 60 s. Known, logged, not yet root-caused.

## Debugging

`/tmp/vmix-macos/<name>/` on the build host: `driver.log`, `serial.log`
(kernel + `VMIX-*` markers), periodic PNG screenshots, `qmp.sock`.
`vmix-run.log` / `system-install.log` from the VMIX volume are printed at the
end of the build. Add `vncDisplay = ":10"` to watch.

## QEMU profile

`helpers/qemu.nix`: q35, `Skylake-Client` CPU spoof (works on AMD),
AppleSMC with the OSK, XHCI keyboard/tablet, AHCI disks, VMware SVGA,
virtio-net pinned to `PciRoot(0x0)/Pci(0x12,0x0)` so OpenCore marks it built-in
(en0, required for Apple ID / iMessage). SMBIOS `MacPro7,1` with four DIMMs
described (avoids the "Memory Modules Misconfigured" warning).

OpenCore comes from OSX-KVM's proven ESP, with Lilu / VirtualSMC /
WhateverGreen replaced by current releases (`upstream.json` → `opencore.kexts`):
the versions OSX-KVM ships disable themselves on macOS 26, and without
VirtualSMC the guest's restart path panics on QEMU's SMC stub
(`SMCWDT smcWriteKey kSMCBadCommand`, nested panic after `MACH Reboot`).
For the same reason QEMU's `isa-applesmc` is not used any more: its presence
makes VirtualSMC step aside ("multiple devices present"); VirtualSMC carries
the OSK itself. Images built before this change still need the stub:
`vmix run --macos --applesmc`. RestrictEvents (`revpatch=memtab`) silences
MacPro7,1's "Memory Modules Misconfigured" at login.

## Guest agent, shares, persistent home, online templates

macOS 13+ ships **Apple's own QEMU guest agent** (`/usr/libexec/AppleQEMUGuestAgent`,
started by launchd when a virtio console port named `org.qemu.guest_agent.0`
appears). It is Apple-signed, needs no approval, and offers `guest-exec` as
root plus `guest-file-*`. vmix uses it everywhere an in-guest agent is needed:

* `vmix run --macos` and the NixOS module attach it by default
  (`/tmp/vmix-qga-<pid>.sock`, `/run/vmix/qga-<name>.sock`); talk to it with any
  QGA client, e.g. `printf '{"execute":"guest-exec","arguments":{"path":"/usr/bin/id","capture-output":true}}\n' | socat - UNIX-CONNECT:<sock>`.
* **online templates** (`bootScript`): `customizeImage` boots the image with the
  agent, runs the script as root (network available, `as_user <cmd>` runs inside
  the logged-in user's session), then shuts down through the agent.
  `templates.software.script { name; script; }`,
  `templates.software.homebrew { formulae; casks; }`,
  `templates.profile.settings { hideWidgets; wallpaper; dockApps; dockAutohide;
  darkMode; showHiddenFiles; }` (wallpaper via the pinned `desktoppr`; Apple
  Events / `osascript` do not work headless — TCC automation consent).
* **offline software templates** run in the PE: `templates.software.pkg { name;
  src; }` (`installer -target`), `templates.software.app { name; src; }`.

`AppleVirtIO.kext` (x86 Tahoe) drives virtio-fs, 9p, block, console, input,
net, sound, balloon, vsock — QEMU's modern virtio-pci devices work as-is:

* **shared folders**: virtio-fs (`virtiofsd` + `vhost-user-fs-pci`, shared
  memory backend). The tag `com.apple.virtio-fs.automount` is mounted by macOS
  itself at `/Volumes/My Shared Files`; further tags are mounted with
  `mount -t virtiofs <tag> <dir>` — the module does that through the guest agent
  for every `shares.<name>` beyond the first. `vmix run --macos --share DIR`.
  (9p does not automount on macOS; the Linux `-virtfs` path is not used.)
* **ephemeral OS disk + persistent home**: `generalize { persistHome = true; }`
  adds `LABEL=vmix-home /Users apfs rw 0 2` to the image's fstab. The host
  provides a virtio-blk disk (`macos.homeDisk` in the module, `--home FILE` in
  the CLI: qcow2/raw file or zvol) that `formatVolume` formats as APFS
  `vmix-home` by booting the PE for ~35 s on first use. macOS mounts it at
  `/Users` before login, home directories are created there; the OS disk can run
  with `snapshot=on` (`disks.os.persist = false`).
* **SPICE**: `-vga vmware` (or `std`) is kept as the display device — macOS has
  no QXL/virtio-gpu driver; USB redirection channels work as for other guests
  (`spice.usbRedir`); there is no vdagent for macOS (no clipboard sharing).
  virtio keyboard/tablet (`AppleVirtIOInput`) are available as
  `qemu.virtioInputArgs` but the USB HID pair is the default.