macOS: drive the install and all customization from a Recovery "PE", no GUI

Replace the screenshot/OCR/keystroke driving of Apple's Recovery with a
"PE": BaseSystem.dmg (a journaled HFS+ volume, writable from Linux) with one
LaunchDaemon added (makeRecoveryPE) that runs /Volumes/VMIX/run.sh as root at
boot, records the status and powers off. launchd loads it alongside its signed
cache (verified on Tahoe 26.6.2); same idea as AutoNBI/Imagr NetBoot images.

- makeImage: the PE runs vmix-install.sh (erase, installer app, SharedSupport
  pkgdmg, startosinstall). Progress is read from the serial console
  (boot-args serial=3 -v, VMIX-* markers) and screenshots (brightness only).
  Fully offline; prepare now takes ~5 min instead of ~10.
- customizeImage: boots the PE with the image attached and runs the template
  offline against the mounted System/Data volumes; OpenCore ScanPolicy
  restricted to HFS+/SATA so only the PE can boot. One PE boot ~30 s. The
  installed macOS is never booted for customization, so nothing depends on
  launchd/BTM approval or a first-boot agent (removed).
- templates rewritten for offline use: generalize creates the user with
  dscl -f (admin, home, auto-login kcpassword, Setup Assistant suppression,
  hostname, locale, timezone, keyboard type, container resize); remote-access,
  no-updates, performance edit the target's plists.
- makeBootDisk: build-time OpenCore variant (serial console, ScanPolicy).
- vm-driver.py rewritten: passive observation only (serial markers, kernel
  boots, panics, brightness), disk+serial-aware hang watchdog, reboot-death
  reset, halt/loginwindow detection. No OCR/tesseract.
- OpenCore: four SMBIOS DIMMs for MacPro7,1 (no "Memory Modules
  Misconfigured" warning).
- tools/soak.sh: repeatability harness.

Verified on daku: base install 23 min end to end; basic + generalize in three
~30 s PE boots; the result auto-logs into the desktop with the created user.

Root cause of the "first-boot hang" (from the serial log): the guest's restart
path panics (IOPlatformHaltRestartAction -> AppleSMC, SMCWDT smcWriteKey
kSMCBadCommand, nested panic) because the pinned OSX-KVM Lilu disables itself
on macOS 26, so VirtualSMC never loads. Handled by the driver (reset within
60 s); kext update to follow.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XsESshRCoBoUVWV9qKURUF
This commit is contained in:
Git Sagar 2026-09-09 11:21:24 -03:00
parent 58a317f5d2
commit 8dc8f4265d
24 changed files with 802 additions and 977 deletions

View file

@ -1,77 +1,59 @@
# Build a pre-installed macOS qcow2 with an unattended QEMU install.
#
# One QEMU session, driven by vm-driver.py:
# Recovery (BaseSystem) boots via OpenCore → driver opens Terminal with
# keystrokes and types "sh /Volumes/VMIX/run.sh" → vmix-install.sh erases the
# disk, rebuilds the installer app from installer-app.tar + the SharedSupport.dmg
# raw disk, runs startosinstall (--installpackage vmix-agent.pkg) → the installer
# reboots through its phases → first boot of macOS runs the vmix agent, which
# executes vmix-run.sh and powers off → QEMU exits.
# Afterwards OpenCore is copied into the image's EFI partition so the result
# boots standalone with plain OVMF. Apply templates with customizeImageFold,
# then .generalize to create the user and set a fresh SMBIOS identity.
{ pkgs, lib, qemu, ident, installerPayload, makeOpenCore, makeVmixVolume, makeAgentPkg, installBootloader, vmixReadback, vmDriver, ... }:
# Build a pre-installed macOS qcow2 with an unattended install driven from the
# vmix PE (Apple's Recovery + one LaunchDaemon, see makeRecoveryPE):
# OpenCore boots the PE → its hook runs /Volumes/VMIX/run.sh (vmix-install.sh)
# → erase the disk, rebuild the installer app from installer-app.tar + the
# SharedSupport raw disk, startosinstall → the installer reboots through its
# phases → the installed system's first boot reaches the loginwindow → the
# driver powers it off. No GUI is driven; progress is read from the serial
# console and screenshots (brightness). Fully offline: no NIC is attached.
# Then OpenCore is copied into the image's own ESP so it boots with plain OVMF.
# Apply templates with customizeImageFold, then .generalize.
{ pkgs, lib, qemu, ident, installerPayload, makeOpenCore, makeBootDisk, makeVmixVolume, installBootloader, vmixReadback, vmDriver, ... }:
{
name ? "macos",
installer, # InstallAssistant.pkg (fetchurl)
recovery, # BaseSystem.dmg (fetchRecovery)
installer, # InstallAssistant.pkg
pe, # makeRecoveryPE output for the same macOS version
diskSize ? "128G",
volumeName ? "Macintosh HD",
smp ? 4,
memSize ? 8192,
cpu ? qemu.defaultCpu,
model ? "MacPro7,1", # SMBIOS model; must be Tahoe-supported (MacPro7,1, iMac20,1/2, MacBookPro16,x)
model ? "MacPro7,1", # SMBIOS model; must be supported by the installed macOS
seed ? name, # MAC address + SystemUUID are derived from this
bootArgs ? "keepsyms=1",
vncDisplay ? null, # e.g. ":10" to watch the install on port 5910
timeout ? 4 * 3600, # seconds for the whole install
extraOpenCoreConfig ? {}, # merged into config.plist
installNetwork ? false, # attach a NIC during install (default: offline — startosinstall
# otherwise hangs on Apple personalization through a flaky NAT)
installNetwork ? false, # attach a NIC during the install (default: offline)
}:
let
mac = ident.macFromSeed seed;
uuid = ident.uuidFromSeed seed;
esp = makeOpenCore { name = "${name}-opencore"; inherit model mac uuid bootArgs; extraConfig = extraOpenCoreConfig; };
esp = makeOpenCore { name = "${name}-opencore"; inherit model mac uuid bootArgs memSize; extraConfig = extraOpenCoreConfig; };
# build-time boot disk: same identity, plus serial console + verbose boot
bootDisk = makeBootDisk { name = "${name}-install"; inherit esp; bootArgs = "${bootArgs} serial=3 -v"; };
payload = installerPayload { inherit name; pkg = installer; };
recoveryImg = pkgs.runCommand "${name}-BaseSystem.img" { nativeBuildInputs = [ pkgs.dmg2img ]; } ''
dmg2img -s ${recovery} $out
'';
agentPkg = makeAgentPkg { };
agentDir = pkgs.runCommand "vmix-agent-files" { } ''
mkdir -p $out
cp ${../guest/agent.sh} $out/agent.sh
cp ${../guest/ch.vmix.agent.plist} $out/ch.vmix.agent.plist
'';
firstBoot = pkgs.writeText "vmix-run.sh" ''
echo "vmix: first boot of the installed system"
sw_vers
exit 0
'';
vmixVol = makeVmixVolume {
inherit name;
size = "512M";
files = [
{ source = ../guest/vmix-install.sh; name = "run.sh"; }
{ source = ../guest/pe-lib.sh; name = "pe-lib.sh"; }
{ source = "${payload}/installer-app.tar"; name = "installer-app.tar"; }
{ source = agentPkg; name = "vmix-agent.pkg"; }
{ source = agentDir; name = "agent"; }
{ source = firstBoot; name = "vmix-run.sh"; }
];
};
driverPython = pkgs.python3.withPackages (p: [ p.pillow p.pytesseract ]);
tesseract = pkgs.tesseract.override { enableLanguages = [ "eng" ]; };
driverPython = pkgs.python3.withPackages (p: [ p.pillow ]);
drv = pkgs.runCommand "${name}-vmix.qcow2" {
__noChroot = true;
requiredSystemFeatures = [ "kvm" ];
nativeBuildInputs = with pkgs; [ pkgs.qemu mtools jq driverPython tesseract libguestfs-with-appliance ];
nativeBuildInputs = with pkgs; [ pkgs.qemu jq driverPython libguestfs-with-appliance ];
} ''
echo "=== vmix: creating ${diskSize} disk ==="
qemu-img create -f qcow2 disk.qcow2 ${diskSize}
# store files are read-only and AHCI needs writable nodes: qcow2 overlays
qemu-img create -q -f qcow2 -F raw -b ${recoveryImg} recovery.qcow2
qemu-img create -q -f qcow2 -F raw -b ${esp}/boot.img ocboot.qcow2
qemu-img create -q -f qcow2 -F raw -b ${pe} pe.qcow2
qemu-img create -q -f qcow2 -F raw -b ${bootDisk}/boot.img ocboot.qcow2
# Apple's postinstall hardlinks the WHOLE InstallAssistant.pkg as
# Contents/SharedSupport/SharedSupport.dmg: the pkg is a "pkgdmg" (xar + koly
@ -99,42 +81,33 @@ let
cp ${pkgs.OVMF.fd}/FV/OVMF_VARS.fd vars.fd
chmod +w vars.fd
VMIX_DISPLAY="-display none"
${lib.optionalString (vncDisplay != null) ''VMIX_DISPLAY="-display none -vnc ${vncDisplay}"''}
${lib.optionalString (vncDisplay == null) ''
VMIX_DF=$(ls -t /tmp/.vmix-display-* 2>/dev/null | head -1)
if [ -n "$VMIX_DF" ] && [ "$(stat -c %s "$VMIX_DF")" -lt 256 ] && ! grep -q -P '[^\x20-\x7e\n]' "$VMIX_DF"; then
export DISPLAY=$(tr -d '\n' < "$VMIX_DF")
export HOME=$(mktemp -d)
export XDG_RUNTIME_DIR=$HOME
export SDL_VIDEODRIVER=x11
VMIX_DISPLAY="-display sdl"
fi
''}
echo "=== vmix: installing ${name} (unattended, 1-2 h; screenshots in /tmp/vmix-macos/${name}) ==="
python3 ${vmDriver} --mode install --name ${name} --timeout ${toString timeout} --progress-file disk.qcow2 -- \
echo "=== vmix: installing ${name} (unattended, ~1 h; logs and screenshots in /tmp/vmix-macos/${name}) ==="
python3 ${vmDriver} --mode install --name ${name} --timeout ${toString timeout} \
--serial-log serial.log --progress-file disk.qcow2 -- \
qemu-system-x86_64 $VMIX_DISPLAY \
${qemu.machineArgs { inherit cpu smp memSize; }} \
${qemu.firmwareArgs "vars.fd"} \
${qemu.serialArgs "serial.log"} \
${qemu.sataDrive { id = "opencore"; port = 0; file = "ocboot.qcow2"; }} \
${qemu.sataDrive { id = "recovery"; port = 1; file = "recovery.qcow2"; }} \
${qemu.sataDrive { id = "pe"; port = 1; file = "pe.qcow2"; }} \
${qemu.sataDrive { id = "system"; port = 2; file = "disk.qcow2"; }} \
${qemu.sataDrive { id = "vmix"; port = 3; file = "vmix.img"; format = "raw"; }} \
${qemu.sataDrive { id = "sharedsupport"; port = 4; file = "sharedsupport.qcow2"; }} \
${lib.optionalString installNetwork (qemu.netArgs { inherit mac; })} \
|| { echo "vmix: install VM failed (see /tmp/vmix-macos/${name})"; exit 1; }
# The driver exits non-zero (handled above) if the install did not reach a
# completed/powered-off state, so reaching here means the OS is installed.
# vmix-run.status is written only when the agent ran (cron/daemon); log it.
# The PE records a status only if run.sh returned, i.e. the install failed
# before the installer took over and rebooted.
${vmixReadback "vmix.img"}
[ "$STATUS" = "0" ] && echo "vmix: first-boot agent completed (status 0)" \
|| echo "vmix: install reached loginwindow (agent status '$STATUS'); image is installed"
if [ -n "$STATUS" ] && [ "$STATUS" != "0" ]; then
echo "vmix: install script failed (status $STATUS), see /tmp/vmix-macos/${name}"; exit 1
fi
${installBootloader { inherit esp; image = "disk.qcow2"; }}
echo "=== vmix: ${name} install complete (serial $(jq -r .serial ${esp}/vmix.json), mac ${mac}) ==="
mv disk.qcow2 $out
'';
in drv // { _vmixOsType = "macos"; macAddress = mac; opencore = esp; inherit model; }
in drv // { _vmixOsType = "macos"; macAddress = mac; opencore = esp; inherit model pe volumeName; }