vmix.nix/lib/images/macos/README.md
Git Sagar e6e2e9f18d macOS README: restart-path bisect result
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XsESshRCoBoUVWV9qKURUF
2026-09-10 13:34:28 -03:00

10 KiB
Raw Blame History

macOS images (Tahoe 26)

Pre-installed, Apple-ID-capable macOS VM images built the same way as the Windows ones: makeImage (unattended install) → templates → .generalize (user, hostname, fresh SMBIOS identity). Runs on QEMU/KVM with OpenCore.

vmix build --image macos.images.tahoe.basic --generalize username=sagar,password=secret,hostname=MAC
vmix run ./result --macos --vnc :10 --mem 8192

Nix: macos.images.tahoe.{pe,upstream,basic,remote} and <image>.generalize { username; password; hostname; timezone; locale; seed; … }.

How it works: the vmix "PE"

Apple's Recovery (BaseSystem.dmg, a plain journaled HFS+ volume) with one LaunchDaemon added (makeRecoveryPE): at boot it mounts a VMIX volume and runs run.sh from it as root, records the exit status and powers off. That is the whole automation surface — the equivalent of Windows PE + Autounattend:

  • no GUI is driven: no OCR, no keystrokes, no screen layouts to learn per macOS version; the hook is a launchd plist, stable across releases (same idea as AutoNBI/Imagr NetBoot images).
  • observable: the guest prints VMIX-* markers to /dev/console, which the build reads from QEMU's serial log (boot-args serial=3 -v). Kernel panics and reboots show up there too. Screenshots are still taken for debugging.
  • offline: no NIC during the install, and the guest blackholes Apple's install/verify endpoints so startosinstall never waits on the network. The only inputs are the pinned InstallAssistant.pkg and BaseSystem.dmg.
  • everything else happens offline from the PE too: templates and generalize mount the image's Data volume (rw) and System volume (ro) and edit them (dscl -f for users, plutil for preferences) — the installed macOS is never booted for customization, so nothing depends on launchd/BTM approval, first-boot agents or auto-login inside the guest. One PE boot ≈ 30 s.

Pipeline

  1. makeRecoveryPE — BaseSystem.dmg → raw HFS+ image + ch.vmix.pe daemon.
  2. makeImage — QEMU with: OpenCore boot disk (build variant with serial console), the PE, the empty target disk, the VMIX volume (vmix-install.sh, installer app skeleton) and the whole InstallAssistant.pkg mapped as a raw disk. The guest script erases the target as APFS, unpacks the app and dds the pkg into it as SharedSupport.dmg (it is a "pkgdmg": xar + koly footer; the bare xar member fails with "pkgdmg is missing a footer"), then runs startosinstall, which reboots itself through the install phases. The installed system's first boot ends at the loginwindow: the driver detects the bright screen and powers the VM down. OpenCore is then copied into the image's own ESP so it boots with plain OVMF.
  3. customizeImage — boots the PE with the image attached (OpenCore ScanPolicy restricted to HFS+ on SATA, so only the PE can boot) and runs the template script with $SYS/$DATA mounted. pe-lib.sh has the helpers.
  4. templates/generalize.nix — user (dscl, admin, home from the user template), auto-login (kcpassword), Setup Assistant suppression, hostname, locale, timezone, keyboard type, container resize, fresh SMBIOS via a new OpenCore ESP (serial/mlb from macserial, MAC + UUID from seed).

Recovery source

recovery.file in upstream.json points at a content-addressed store path for the verified Tahoe BaseSystem.dmg (Apple's CDN load-balances Sequoia/Tahoe during the rollout, so a plain fetch is non-deterministic). Reproduce it on any host with nix store add-path --name macos-tahoe-BaseSystem.dmg BaseSystem.dmg. Drop recovery.file to fetch from Apple instead (fetchRecovery retries until the pinned hash matches).

Reliability

Things QEMU does intermittently, and what handles each (all in vm-driver.py and vmix-install.sh; every event is logged with a reason):

  • startosinstall prepare stalls or crawls — the guest kills and retries it on a freshly erased target (free-space watchdog + time cap).
  • the installer comes back to the PE instead of the install phase — the PE counts boots and simply re-runs the install (max 3).
  • the installed system hangs at the Apple logo on first boot — a system_reset is issued only when the screen is dark and frozen and disk and serial console are idle, so a slow-but-working boot is never interrupted.
  • macOS shutdown -h halts to a black screen without an ACPI power-off — an idle black screen counts as a completed halt.
  • a kernel panic (seen on the serial console) resets the VM.
  • a wedged run fails at the 4 h timeout instead of hanging.

tools/soak.sh <flake> macos.images.tahoe.upstream 3 rebuilds an image N times and tabulates outcome, duration, boots, resets, panics and retries. Measured 2026-09-09 on the build host (Ryzen 7 7840HS, ZFS), Tahoe 26.6.2, VirtualSMC-only, PE install — 3 of 3 builds completed:

run minutes kernel boots prepare tries panics (self-recovered) reboot deaths
1 30 8 1 2 0
2 26 7 1 1 0
3 26 7 1 1 0

What still happens: at roughly one in ten guest-initiated reboots the guest either panics (GPF in launchd/kernel_task context shortly after MACH Reboot or within the first 15 s of the next boot — tmpfs/APFS/zone corruption signatures, i.e. memory or register state, not one driver) or never comes back (dead after IOPlatformHaltRestartAction). XNU reboots itself after a panic; the driver resets a dead guest after 60 s, so builds complete. A device bisect (tools-style 1030 PE reboots per variant: VMware SVGA vs std VGA, no HDA, EHCI input, 1 vCPU) showed the rate is independent of the emulated devices and of SMP; Haswell-noTSX does not boot Tahoe. Host: AMD Zen 4, kvm_amd, Intel Skylake-Client vCPU model — the FPU-context-switch panic points at XSAVE state handling on that combination. Not fixed; a vmix run VM that hangs on Restart must be reset from the host.

Debugging

/tmp/vmix-macos/<name>/ on the build host: driver.log, serial.log (kernel + VMIX-* markers), periodic PNG screenshots, qmp.sock. vmix-run.log / system-install.log from the VMIX volume are printed at the end of the build. Add vncDisplay = ":10" to watch.

QEMU profile

helpers/qemu.nix: q35, Skylake-Client CPU spoof (works on AMD), AppleSMC with the OSK, XHCI keyboard/tablet, AHCI disks, VMware SVGA, virtio-net pinned to PciRoot(0x0)/Pci(0x12,0x0) so OpenCore marks it built-in (en0, required for Apple ID / iMessage). SMBIOS MacPro7,1 with four DIMMs described (avoids the "Memory Modules Misconfigured" warning).

OpenCore comes from OSX-KVM's proven ESP, with Lilu / VirtualSMC / WhateverGreen replaced by current releases (upstream.jsonopencore.kexts): the versions OSX-KVM ships disable themselves on macOS 26, and without VirtualSMC the guest's restart path panics on QEMU's SMC stub (SMCWDT smcWriteKey kSMCBadCommand, nested panic after MACH Reboot). For the same reason QEMU's isa-applesmc is not used any more: its presence makes VirtualSMC step aside ("multiple devices present"); VirtualSMC carries the OSK itself. Images built before this change still need the stub: vmix run --macos --applesmc. RestrictEvents (revpatch=memtab) silences MacPro7,1's "Memory Modules Misconfigured" at login.

Guest agent, shares, persistent home, online templates

macOS 13+ ships Apple's own QEMU guest agent (/usr/libexec/AppleQEMUGuestAgent, started by launchd when a virtio console port named org.qemu.guest_agent.0 appears). It is Apple-signed, needs no approval, and offers guest-exec as root plus guest-file-*. vmix uses it everywhere an in-guest agent is needed:

  • vmix run --macos and the NixOS module attach it by default (/tmp/vmix-qga-<pid>.sock, /run/vmix/qga-<name>.sock); talk to it with any QGA client, e.g. printf '{"execute":"guest-exec","arguments":{"path":"/usr/bin/id","capture-output":true}}\n' | socat - UNIX-CONNECT:<sock>.
  • online templates (bootScript): customizeImage boots the image with the agent, runs the script as root (network available, as_user <cmd> runs inside the logged-in user's session), then shuts down through the agent. templates.software.script { name; script; }, templates.software.homebrew { formulae; casks; }, templates.profile.settings { hideWidgets; wallpaper; dockApps; dockAutohide; darkMode; showHiddenFiles; } (wallpaper via the pinned desktoppr; Apple Events / osascript do not work headless — TCC automation consent).
  • offline software templates run in the PE: templates.software.pkg { name; src; } (installer -target), templates.software.app { name; src; }.

AppleVirtIO.kext (x86 Tahoe) drives virtio-fs, 9p, block, console, input, net, sound, balloon, vsock — QEMU's modern virtio-pci devices work as-is:

  • shared folders: virtio-fs (virtiofsd + vhost-user-fs-pci, shared memory backend). The tag com.apple.virtio-fs.automount is mounted by macOS itself at /Volumes/My Shared Files; further tags are mounted with mount -t virtiofs <tag> <dir> — the module does that through the guest agent for every shares.<name> beyond the first. vmix run --macos --share DIR. (9p does not automount on macOS; the Linux -virtfs path is not used.)
  • ephemeral OS disk + persistent home: generalize { persistHome = true; } gives the account its home directory on an APFS volume labelled vmix-home (NFSHomeDirectory = /Volumes/vmix-home/<user>; macOS refuses mounts over /Users, which is a firmlink). The host provides a virtio-blk disk (macos.homeDisk in the module, --home FILE in the CLI: qcow2/raw file or zvol) that formatVolume formats as APFS vmix-home by booting the PE for ~35 s on first use; diskarbitrationd mounts it before login and loginwindow creates the home directory there on first login. The OS disk can then run with snapshot=on (disks.os.persist = false).
  • SPICE: -vga vmware (or std) is kept as the display device — macOS has no QXL/virtio-gpu driver; USB redirection channels work as for other guests (spice.usbRedir); there is no vdagent for macOS (no clipboard sharing). virtio keyboard/tablet (AppleVirtIOInput) are available as qemu.virtioInputArgs but the USB HID pair is the default.