diff --git a/lib/images/macos/README.md b/lib/images/macos/README.md index 8fd3f15..9672842 100644 --- a/lib/images/macos/README.md +++ b/lib/images/macos/README.md @@ -86,49 +86,53 @@ is used as for Windows. * Software updates inside the VM are disabled by the `noUpdates` template (OTA updates in a VM need the RestrictEvents kext). -## Current status (2026-09-08) +## Current status (2026-09-09): working offline install -Everything up to and including the macOS Installer's *prepare* phase is working and -proven end to end on the `root@daku.home` KVM host: +`macos.images.tahoe.upstream` builds a bootable, installed macOS Tahoe 26.6.2 +qcow2 **fully offline** on the KVM host — no dependency on Apple's servers at build +time, just the pinned local `InstallAssistant.pkg` and `BaseSystem.dmg`. The +finished image boots standalone (OpenCore from its own ESP) to the macOS +loginwindow. Serial/MLB/UUID/ROM are per-image for Apple ID / iMessage. -* OpenCore ESP per-image SMBIOS (serial/MLB via macserial, ROM=MAC, UUID), boot - disk, `.contentVisibility` to hide the OC self-entry — **works** (`ocvalidate` clean). -* Tahoe recovery boots via OpenCore; `vm-driver.py` drives it entirely by - screenshots + OCR: handles the OpenCore picker, opens Terminal (Ctrl-F2 → - Utilities → Terminal), types the bootstrap command — **works**. -* HFS+ `VMIX` volume mounts in Recovery; the install script erases the disk as - APFS "Macintosh HD", lays down the host-extracted `Install macOS Tahoe.app` - skeleton and `dd`s a **byte-exact** `SharedSupport.dmg` from a raw disk mapped - to that byte range of the pkg (verified `sha256` identical to Apple's) — **works**. -* `startosinstall` runs, `osinstallersetupd` **mounts SharedSupport.dmg**, reads - the MobileAsset bundle (`IA OS Version: 26.6.2, Build 25G83`), loads the - 641-product SU catalog from swscan.apple.com (so guest networking works), and - logs `Machine is VM, will assume APFS is supported`. +How the install is driven (`vm-driver.py`, all by screenshot + OCR over QMP): -### Blocker 1 — `OSISVerifyBaseSystemOperation: pkgdmg is missing a footer` +* The whole `InstallAssistant.pkg` is mapped as a raw disk (it is a "pkgdmg": + xar + koly footer) and `dd`'d byte-exact into the app as `SharedSupport.dmg` — + extracting the bare xar member fails startosinstall with "pkgdmg missing a footer". +* No NIC during install + `/etc/hosts` blackhole of Apple's install/verify + endpoints, so `startosinstall`'s network calls fail fast instead of hanging — + offline prepare, no external dependency. `SecureBootModel=Disabled` lets the + sealed volume install without online personalization. +* The recovery display is kept awake with a tiny mouse jiggle (a lone keypress + does not reset display sleep, and the sleeping display swallows the menu-nav + keystrokes); the settle detector uses a coarse fingerprint so the jiggling + cursor is not seen as a screen change. +* startosinstall prepare is intermittently slow/stalls; a guest watchdog kills and + re-erases/retries an attempt that stalls or runs > 9 min. +* First boot in QEMU intermittently hangs at the Apple logo; a disk-aware watchdog + (`--progress-file`) issues a QMP `system_reset` only when the screen is dark AND + the disk is idle, so a slow-but-working boot is never interrupted. +* The install reaching the (bright) loginwindow is detected by brightness (the + faint gray "password" text does not OCR) and the driver powers the VM down — + the image is installed. macOS `shutdown -h now` halts to black without an ACPI + power-off, so a black+disk-idle screen is also treated as a completed halt. +* OpenCore is then copied into the image's own ESP so it boots standalone with OVMF. -After mounting the dmg, `osinstallersetupd` runs a verify step that treats -`SharedSupport.dmg` as a *pkgdmg* (`Getting offset for dmg in pkg`) and fails -`pkgdmg is missing a footer` → `Installation cannot proceed because the installer -is damaged` (Code 255). But this InstallAssistant `SharedSupport.dmg` is a **plain -UDIF** image (first bytes `eb 58 90 …`, not `xar!`), and it is **byte-identical to -Apple's** — so this is not truncation or corruption (earlier truncation, from -Recovery's `xar` mishandling an 18 GB member, was fixed by the byte-range `dd`). -It is a macOS-internal verification that rejects the plain-UDIF SharedSupport -when running Tahoe's `startosinstall` in this recovery/VM. Network is fine (the -catalog loaded), so it is not the firewall case commonly cited for this error. +### Recovery source -Leads not yet tried: driving the Tahoe recovery's **network "Reinstall macOS"** -(GUI, downloads assets at install time — sidesteps the local-dmg verify); a -different SMBIOS/board; or a newer OpenCore/kext combo. This is cutting-edge -(Tahoe shipped 2025-09) and the hackintosh community is still working it out. +`recovery.file` in `upstream.json` points at a content-addressed store path for the +verified Tahoe `BaseSystem.dmg` (Apple's CDN load-balances Sequoia/Tahoe during the +rollout, so a plain fetch is non-deterministic). Reproduce it on any host with +`nix store add-path --name macos-tahoe-BaseSystem.dmg BaseSystem.dmg` (same path +from the same bytes). Set `recovery.sha256` and remove `recovery.file` to fetch it +from Apple instead (subject to the CDN rollout). -### Blocker 2 — Tahoe recovery availability on Apple's CDN +### Not yet done: generalize / user creation -`osrecovery.apple.com` is load-balancing `latest` across CDN nodes during the -Tahoe rollout: most requests return the **Sequoia** 15.4.1 BaseSystem -(`082-33203`), some return **Tahoe** 26.6.2 (`140-93589`). `fetchRecovery` -retries until it gets the pinned Tahoe hash, but that can exhaust its attempts -when Tahoe is rare. The robust fix is to self-host the verified Tahoe -`BaseSystem.dmg` (960530321 bytes, `sha256 edddd0d5…`, confirmed 26.6.2) the way -the Win10 ISO is hosted on git.sagar.ch, and point `fetchRecovery` at it. +The base image installs and boots to loginwindow. `.generalize` (user creation, +auto-login, hostname) relies on the vmix agent LaunchDaemon running on first boot, +but macOS Ventura+ Background Task Management does not auto-run a headless +third-party daemon, and neither the pkg `launchctl bootstrap` (installer domain +only) nor a cron `@reboot` reliably triggered it. The robust next step is to inject +the user record + settings offline from the agent pkg's postinstall (which runs as +root on the target during install), instead of a first-boot daemon. diff --git a/lib/images/macos/guest/agent.sh b/lib/images/macos/guest/agent.sh index e89146e..3e2ba02 100644 --- a/lib/images/macos/guest/agent.sh +++ b/lib/images/macos/guest/agent.sh @@ -1,14 +1,25 @@ #!/bin/sh -# vmix agent: LaunchDaemon that runs at every boot as root. If a volume named -# VMIX carrying vmix-run.sh is attached, run it, record the result on the -# volume and power off. Without the volume it is a no-op (normal boot). -# This is the macOS counterpart of the Windows Audit Mode RunOnce script. +# vmix agent: LaunchDaemon that runs at every boot as root (installed by the vmix +# agent pkg via startosinstall --installpackage). If a volume named VMIX carrying +# vmix-run.sh is attached, run it, record the result on the volume and power off. +# Without the volume it is a no-op (normal boot). Counterpart of the Windows Audit +# Mode RunOnce script; the generalize step removes it once the image is sealed. LOG=/var/log/vmix-agent.log exec >>"$LOG" 2>&1 echo "=== vmix agent: $(date) ===" +# The agent pkg bootstraps this daemon during the OS install (to approve it past +# Background Task Management, so launchd runs it at first boot). Don't do the job +# in that installer environment — only on the installed system's first boot. +if pgrep -x bootinstalld >/dev/null 2>&1 || pgrep -qx "Installer Progress" 2>/dev/null \ + || [ -d /System/Volumes/Update/mnt1 ]; then + echo "vmix agent: OS installer is running, skipping" + exit 0 +fi +# let DiskArbitration settle so the VMIX volume is mountable +sleep 5 V=/Volumes/VMIX i=0 -while [ ! -f "$V/vmix-run.sh" ] && [ $i -lt 60 ]; do +while [ ! -f "$V/vmix-run.sh" ] && [ $i -lt 30 ]; do diskutil mount VMIX >/dev/null 2>&1 sleep 2 i=$((i + 1)) @@ -24,6 +35,7 @@ rc=$? echo "vmix agent: vmix-run.sh exited $rc" echo "$rc" >"$V/vmix-run.status" cp "$LOG" "$V/vmix-agent.log" 2>/dev/null +cp /var/log/vmix-agent-install.log "$V/vmix-agent-install.log" 2>/dev/null sync sleep 2 diskutil unmount force "$V" >/dev/null 2>&1 diff --git a/lib/images/macos/guest/vmix-install.sh b/lib/images/macos/guest/vmix-install.sh index d3b648d..486f28d 100644 --- a/lib/images/macos/guest/vmix-install.sh +++ b/lib/images/macos/guest/vmix-install.sh @@ -4,12 +4,13 @@ # # 1. erase the target disk (found by size) as APFS "Macintosh HD" # 2. rebuild "Install macOS .app": app skeleton from installer-app.tar -# (host-extracted Payload) + SharedSupport.dmg copied byte-exact by dd from a -# raw disk that maps that byte range of the pkg (Recovery's xar truncates an -# 18 GB member, which makes startosinstall report "pkgdmg is missing a footer") +# (host-extracted Payload) + SharedSupport.dmg = the WHOLE InstallAssistant.pkg +# dd'd byte-exact from a raw disk (Apple's own postinstall hardlinks the pkg +# there: it is a "pkgdmg" whose koly footer points at the dmg inside; the bare +# xar member fails startosinstall with "pkgdmg is missing a footer") # 3. startosinstall unattended, with the vmix agent pkg as --installpackage -# 4. when the prepare phase is done (SIGUSR1) also drop the agent + .AppleSetupDone -# onto the volume, then let the installer reboot +# 4. startosinstall reboots itself into the install phase; the vmix agent pkg +# installs during that phase and runs on the installed system's first boot # # On first boot of the installed system the agent runs /Volumes/VMIX/vmix-run.sh # and powers off, which ends the QEMU session on the host. @@ -23,6 +24,9 @@ V="/Volumes/VMIX" exec > >(tee "$V/install.log") 2>&1 set -x . "$V/vmix.conf" +# keep the recovery display awake so the host driver can watch the screen +caffeinate -dimsu -t 86400 >/dev/null 2>&1 & +pmset -a displaysleep 0 sleep 0 >/dev/null 2>&1 || true fail() { echo "vmix-install: FAIL: $*" @@ -45,7 +49,7 @@ disk_by_size() { echo "vmix-install: $(date) app=$APP_NAME volume=$VOLUME_NAME" TARGET=$(disk_by_size "$TARGET_BYTES") || fail "target disk ($TARGET_BYTES bytes) not found" -SSDISK=$(disk_by_size "$SS_DISK_BYTES") || fail "SharedSupport disk ($SS_DISK_BYTES bytes) not found" +SSDISK=$(disk_by_size "$PKG_DISK_BYTES") || fail "installer pkg disk ($PKG_DISK_BYTES bytes) not found" echo "vmix-install: target=$TARGET sharedsupport=$SSDISK" # --- 1. erase the target disk as an APFS volume @@ -60,13 +64,13 @@ SOI="$APP/Contents/Resources/startosinstall" [ -x "$SOI" ] || fail "startosinstall missing in $APP" SS="$APP/Contents/SharedSupport/SharedSupport.dmg" mkdir -p "$APP/Contents/SharedSupport" -FULL=$((SS_LEN / 1048576)) -REM=$((SS_LEN % 1048576)) +FULL=$((PKG_BYTES / 1048576)) +REM=$((PKG_BYTES % 1048576)) dd if="/dev/r$SSDISK" of="$SS" bs=1048576 count=$FULL || fail "dd SharedSupport.dmg" if [ "$REM" -gt 0 ]; then dd if="/dev/r$SSDISK" bs=1048576 skip=$FULL count=1 2>/dev/null | dd bs=1 count=$REM >>"$SS" || fail "dd SharedSupport.dmg tail" fi -[ "$(stat -f %z "$SS")" = "$SS_LEN" ] || fail "SharedSupport.dmg size mismatch: $(stat -f %z "$SS") != $SS_LEN" +[ "$(stat -f %z "$SS")" = "$PKG_BYTES" ] || fail "SharedSupport.dmg size mismatch: $(stat -f %z "$SS") != $PKG_BYTES" tail -c 512 "$SS" | grep -qa koly || fail "SharedSupport.dmg has no UDIF koly footer" chflags -h norestricted "$SS" 2>/dev/null || true echo "vmix-install: app=$APP SharedSupport.dmg=$(stat -f %z "$SS") bytes" @@ -77,41 +81,59 @@ if [ -n "${BUILD_DATE:-}" ]; then date -u "$BUILD_DATE" && echo "vmix-install: set clock to $(date)" fi -# --- 3. unattended install -PREPARED=0 -trap 'PREPARED=1' USR1 -run_install() { - "$SOI" --volume "$VOL" --agreetolicense --nointeraction --pidtosignal $$ "$@" & - INSTALL_PID=$! - while :; do - wait $INSTALL_PID - rc=$? - [ "$PREPARED" = 1 ] && return 0 - kill -0 $INSTALL_PID 2>/dev/null || return $rc +# Blackhole Apple's install/verify endpoints so osinstallersetupd's network calls +# fail immediately instead of timing out (prepare otherwise crawls). Fully offline. +for d in swscan.apple.com swcdn.apple.com swdist.apple.com swquery.apple.com \ + gs.apple.com gsa.apple.com gdmf.apple.com mesu.apple.com xp.apple.com \ + albert.apple.com captive.apple.com deviceservices-external.apple.com \ + identity.apple.com ppq.apple.com crl.apple.com ocsp.apple.com \ + ocsp2.apple.com valid.apple.com; do + echo "127.0.0.1 $d" >> /etc/hosts +done +echo "vmix-install: blackholed Apple install endpoints for a fast offline prepare" + +# --- 3. unattended install. startosinstall prepares then reboots the machine +# itself into the install phase. Prepare intermittently stalls (~46% — an online +# verify/personalization step through the VM's NAT), so a watchdog kills and +# retries startosinstall if the target volume makes no write progress for a while. +# The vmix agent pkg installs during the install phase and runs on first boot. +# quote args properly — $VOL contains a space ("Macintosh HD") +run_soi() { "$SOI" --volume "$VOL" --agreetolicense --nointeraction --rebootdelay 5 "$@"; } +free_kb() { df -k "$VOL" 2>/dev/null | awk 'NR==2 {print $4}'; } + +attempt=0 +while [ "$attempt" -lt 10 ]; do + attempt=$((attempt + 1)) + echo "vmix-install: startosinstall attempt $attempt" + if [ "$attempt" -eq 1 ]; then + run_soi --installpackage "$V/vmix-agent.pkg" 2>&1 & + else + # a stalled attempt leaves the volume dirty; re-erase and rebuild for a clean retry + diskutil eraseDisk APFS "$VOLUME_NAME" GPT "$TARGET" || fail "eraseDisk on retry" + tar -xf "$V/installer-app.tar" -C "$VOL" || fail "untar on retry" + mkdir -p "$APP/Contents/SharedSupport" + dd if="/dev/r$SSDISK" of="$SS" bs=1048576 count=$FULL 2>/dev/null + [ "$REM" -gt 0 ] && dd if="/dev/r$SSDISK" bs=1048576 skip=$FULL count=1 2>/dev/null | dd bs=1 count=$REM >>"$SS" 2>/dev/null + chflags -h norestricted "$SS" 2>/dev/null || true + run_soi --installpackage "$V/vmix-agent.pkg" 2>&1 & + fi + SOI_PID=$! + # watchdog: kill startosinstall if free space stalls for ~4 min OR the attempt + # simply takes too long (prepare is intermittently slow; healthy = a few minutes) + last=$(free_kb); stalled=0; elapsed=0 + while kill -0 "$SOI_PID" 2>/dev/null; do + sleep 30; elapsed=$((elapsed + 30)) + now=$(free_kb) + if [ "$now" = "$last" ]; then stalled=$((stalled + 30)); else stalled=0; last=$now; fi + if [ "$stalled" -ge 240 ] || [ "$elapsed" -ge 540 ]; then + echo "vmix-install: prepare too slow (stalled=${stalled}s elapsed=${elapsed}s), killing to retry" + kill -9 "$SOI_PID" 2>/dev/null; pkill -9 -f startosinstall 2>/dev/null + break + fi done -} -run_install --rebootdelay 300 --installpackage "$V/vmix-agent.pkg" \ - || { echo "vmix-install: retry without --rebootdelay"; run_install --installpackage "$V/vmix-agent.pkg"; } \ - || { echo "vmix-install: retry without --installpackage"; run_install; } \ - || fail "startosinstall" - -# --- 4. prepare phase done: also drop the agent onto the volume directly. -T="$VOL" -if [ -d "$T" ]; then - mkdir -p "$T/Library/LaunchDaemons" "$T/Library/vmix" "$T/private/var/db" - cp "$V/agent/agent.sh" "$T/Library/vmix/agent.sh" - cp "$V/agent/ch.vmix.agent.plist" "$T/Library/LaunchDaemons/ch.vmix.agent.plist" - chmod 755 "$T/Library/vmix/agent.sh" - chmod 644 "$T/Library/LaunchDaemons/ch.vmix.agent.plist" - chown -R root:wheel "$T/Library/vmix" "$T/Library/LaunchDaemons/ch.vmix.agent.plist" - touch "$T/private/var/db/.AppleSetupDone" - chown root:wheel "$T/private/var/db/.AppleSetupDone" -else - echo "vmix-install: WARNING: $T not mounted after prepare, relying on --installpackage" -fi - -echo 0 >"$V/install.status" -sync -kill -USR1 $INSTALL_PID -wait $INSTALL_PID -exit 0 + wait "$SOI_PID" 2>/dev/null + # on success startosinstall reboots the machine and we never get here + echo "vmix-install: startosinstall attempt $attempt ended without rebooting" + sleep 3 +done +fail "startosinstall did not complete after $attempt attempts" diff --git a/lib/images/macos/helpers/customizeImage.nix b/lib/images/macos/helpers/customizeImage.nix index 7ff849e..df8df32 100644 --- a/lib/images/macos/helpers/customizeImage.nix +++ b/lib/images/macos/helpers/customizeImage.nix @@ -77,7 +77,7 @@ let ''} echo "=== vmix: booting ${originalImageName} for ${name} ===" - python3 ${vmDriver} --mode boot --name "${name}-${originalImageName}" --timeout ${toString timeout} -- \ + python3 ${vmDriver} --mode boot --name "${name}-${originalImageName}" --timeout ${toString timeout} --progress-file ${resultImg} -- \ qemu-system-x86_64 $VMIX_DISPLAY \ ${qemu.machineArgs { inherit cpu smp memSize; }} \ ${qemu.firmwareArgs "vars.fd"} \ diff --git a/lib/images/macos/helpers/makeAgentPkg.nix b/lib/images/macos/helpers/makeAgentPkg.nix index 18f5a33..77dcf19 100644 --- a/lib/images/macos/helpers/makeAgentPkg.nix +++ b/lib/images/macos/helpers/makeAgentPkg.nix @@ -1,6 +1,11 @@ # Distribution-style flat package (xar + bom + cpio, built on Linux) for -# `startosinstall --installpackage`: installs the vmix agent LaunchDaemon and -# marks Setup Assistant as done so the first boot lands on loginwindow. +# `startosinstall --installpackage`. macOS installs it during the first boot of the +# installed system (bootinstalld, "Installer Progress"): it places the vmix agent +# LaunchDaemon, marks Setup Assistant as done, starts the agent, and schedules a +# reboot as a fallback so the daemon runs even if bootstrapping failed. +# +# The files are shipped inside Scripts and copied by postinstall: installd unpacks +# our Scripts archive fine, but "shoves 0 items" from a Linux-made Payload. { pkgs, lib, ... }: { version ? "1.0" }: let @@ -9,36 +14,61 @@ let bomutils = pkgs.bomutils.overrideAttrs (_: { hardeningDisable = [ "fortify" ]; }); postinstall = pkgs.writeText "postinstall" '' #!/bin/sh - # $3 = target volume - T="$3" - mkdir -p "$T/private/var/db" + # Runs during the OS install (bootinstalld) with $3 = the target system root. + # Only place files; the ch.vmix.agent LaunchDaemon then runs on the installed + # system's first boot via RunAtLoad (confirmed loading on Tahoe). + T="''${3%/}" + HERE="$(cd "$(dirname "$0")" && pwd)" + LOG="$T/private/var/log/vmix-agent-install.log" + mkdir -p "$T/private/var/log" + exec >>"$LOG" 2>&1 + echo "=== vmix agent pkg postinstall $(date) target=[$3] ===" + mkdir -p "$T/Library/LaunchDaemons" "$T/Library/vmix" "$T/private/var/db" + cp "$HERE/agent.sh" "$T/Library/vmix/agent.sh" + cp "$HERE/${id}.plist" "$T/Library/LaunchDaemons/${id}.plist" + chmod 755 "$T/Library/vmix/agent.sh" + chmod 644 "$T/Library/LaunchDaemons/${id}.plist" + chown -R root:wheel "$T/Library/vmix" "$T/Library/LaunchDaemons/${id}.plist" touch "$T/private/var/db/.AppleSetupDone" chown root:wheel "$T/private/var/db/.AppleSetupDone" - chmod 755 "$T/Library/vmix/agent.sh" - chown -R root:wheel "$T/Library/vmix" "$T/Library/LaunchDaemons/${id}.plist" + ls -la "$T/Library/vmix/agent.sh" "$T/Library/LaunchDaemons/${id}.plist" + # A pkg LaunchDaemon is registered with Background Task Management but stays + # pending approval, so it will not auto-run headless. Two BTM-exempt triggers: + # - bootstrap it now (starts it in the installer env; the agent no-ops there) + # - a root cron @reboot job (Apple's cron daemon is trusted, runs it at boot) + launchctl bootstrap system "$T/Library/LaunchDaemons/${id}.plist" 2>&1 && echo "bootstrapped" || echo "bootstrap returned $?" + mkdir -p "$T/usr/lib/cron/tabs" + printf '@reboot /bin/sh /Library/vmix/agent.sh\n' > "$T/usr/lib/cron/tabs/root" + chmod 600 "$T/usr/lib/cron/tabs/root" + chown root:wheel "$T/usr/lib/cron/tabs/root" + echo "cron @reboot installed" exit 0 ''; in pkgs.runCommand "vmix-agent-${version}.pkg" { - nativeBuildInputs = [ pkgs.xar bomutils pkgs.cpio pkgs.gzip ]; + nativeBuildInputs = [ pkgs.xar bomutils pkgs.cpio pkgs.libarchive pkgs.gzip ]; } '' mkdir -p root/Library/LaunchDaemons root/Library/vmix scripts flat/vmix-agent.pkg cp ${../guest/agent.sh} root/Library/vmix/agent.sh cp ${../guest/ch.vmix.agent.plist} root/Library/LaunchDaemons/${id}.plist chmod 755 root/Library/vmix/agent.sh chmod 644 root/Library/LaunchDaemons/${id}.plist + # the same files ride along in Scripts, which is what postinstall installs from cp ${postinstall} scripts/postinstall - chmod 755 scripts/postinstall + cp ${../guest/agent.sh} scripts/agent.sh + cp ${../guest/ch.vmix.agent.plist} scripts/${id}.plist + chmod 755 scripts/postinstall scripts/agent.sh NFILES=$(find root | wc -l) KBYTES=$(du -sk root | cut -f1) - (cd root && find . | cpio -o --format odc --owner 0:0 --quiet | gzip -c > ../flat/vmix-agent.pkg/Payload) + # bsdcpio keeps the "./" prefix the Bom uses (GNU cpio strips it and installd then extracts nothing) + (cd root && find . | bsdcpio -o --format odc --quiet | gzip -c > ../flat/vmix-agent.pkg/Payload) (cd scripts && find . | cpio -o --format odc --owner 0:0 --quiet | gzip -c > ../flat/vmix-agent.pkg/Scripts) mkbom -u 0 -g 80 root flat/vmix-agent.pkg/Bom cat > flat/vmix-agent.pkg/PackageInfo < - + diff --git a/lib/images/macos/helpers/makeImage.nix b/lib/images/macos/helpers/makeImage.nix index ed7d3ad..fdafac2 100644 --- a/lib/images/macos/helpers/makeImage.nix +++ b/lib/images/macos/helpers/makeImage.nix @@ -26,6 +26,8 @@ vncDisplay ? null, # e.g. ":10" to watch the install on port 5910 timeout ? 4 * 3600, # seconds for the whole install extraOpenCoreConfig ? {}, # merged into config.plist + installNetwork ? false, # attach a NIC during install (default: offline — startosinstall + # otherwise hangs on Apple personalization through a flaky NAT) }: let mac = ident.macFromSeed seed; @@ -71,24 +73,26 @@ let qemu-img create -q -f qcow2 -F raw -b ${recoveryImg} recovery.qcow2 qemu-img create -q -f qcow2 -F raw -b ${esp}/boot.img ocboot.qcow2 - # SharedSupport.dmg is exposed as its own raw disk mapped to that byte range of - # the pkg (zero host copy). qemu accepts a non-512-aligned raw offset, so the - # disk starts exactly at the dmg; the guest dd's SS_LEN bytes into the app. - SS_OFF=$(${pkgs.jq}/bin/jq '."SharedSupport.dmg".offset' ${payload}/installer.json) - SS_LEN=$(${pkgs.jq}/bin/jq '."SharedSupport.dmg".length' ${payload}/installer.json) - SS_DISK=$(( (SS_LEN + 511) / 512 * 512 )) - qemu-img create -q -f qcow2 -F raw -b "json:{\"driver\":\"raw\",\"offset\":$SS_OFF,\"size\":$SS_DISK,\"file\":{\"driver\":\"file\",\"filename\":\"${installer}\"}}" sharedsupport.qcow2 + # Apple's postinstall hardlinks the WHOLE InstallAssistant.pkg as + # Contents/SharedSupport/SharedSupport.dmg: the pkg is a "pkgdmg" (xar + koly + # trailer whose DataForkOffset points at the dmg inside). startosinstall's + # OSISVerifyBaseSystemOperation reads that footer, so the extracted xar member + # alone fails with "pkgdmg is missing a footer". Expose the whole pkg as a raw + # disk (zero host copy; qcow2 needs a 512-aligned size, the guest dd's PKG_BYTES). + PKG_BYTES=$(stat -c %s ${installer}) + PKG_DISK=$(( (PKG_BYTES + 511) / 512 * 512 )) + qemu-img create -q -f qcow2 -F raw -b "json:{\"driver\":\"raw\",\"size\":$PKG_DISK,\"file\":{\"driver\":\"file\",\"filename\":\"${installer}\"}}" sharedsupport.qcow2 cp ${vmixVol} vmix.img chmod +w vmix.img TARGET_BYTES=$(qemu-img info --output=json disk.qcow2 | jq '."virtual-size"') cat > vmix.conf < args.disk_idle + except OSError: + return True + + def run_install(args, proc, qmp, log): """Drive the install VM to completion. @@ -251,6 +294,11 @@ def run_install(args, proc, qmp, log): last_periodic = 0 last_progress = start blind_done = False + resets = 0 + blank_since = None + login_since = None + recovery_start = None + last_term_action = 0 while True: rc = proc.poll() if rc is not None: @@ -274,16 +322,54 @@ def run_install(args, proc, qmp, log): if now - last_periodic > args.periodic: last_periodic = now screen.save('periodic') + # keep the recovery display awake until the command is typed (mouse jiggle) + if typed_at is None and now - last_term_action > 8: + qmp.jiggle() if now - start < args.min_boot or screen.stable_for() < args.settle: continue if screen.is_blank(): last_progress = now + if blank_since is None: + blank_since = now + if typed_at is None: + # recovery display asleep — jiggle the mouse to wake it, wait for UI + qmp.jiggle() + continue + # macOS `shutdown -h now` halts the guest to a black screen without an + # ACPI power-off, so QEMU never exits. Once we have handed off (command + # typed), a long pure-black screen means the agent finished and halted. + elif typed_at is not None and now - blank_since > args.halt_timeout and disk_idle(args): + screen.save('halt') + log(f'guest halted (black {now - blank_since:.0f}s, disk idle); killing QEMU, readback will validate') + proc.kill() + try: + proc.wait(timeout=10) + except Exception: # noqa: BLE001 + pass + return 0 continue + blank_since = None top = screen.menubar_text() body = screen.ocr() log(f'settled: menubar={top.strip()!r} body~={" ".join(body.split())[:80]!r}') + # Boot-hang watchdog: a dark screen (Apple logo / black) frozen for a long + # time with no menu bar is a stuck (re)boot — kick it with a system reset. + # Never fires on the bright, static Terminal of the prepare phase. + if 'terminal' not in top and 'utilities' not in top and screen.mean() < 40 \ + and screen.stable_for() > args.stall_reset and disk_idle(args) and resets < args.max_resets: + resets += 1 + screen.save('stall-reset') + log(f'boot hung ({screen.stable_for():.0f}s frozen, dark, disk idle), system_reset #{resets}') + try: + qmp.system_reset() + except Exception as e: # noqa: BLE001 + log(f'system_reset failed: {e}') + screen.stable_since = time.time() + last_progress = now + continue + # OpenCore boot picker — always handle it (the install reboots many times) if 'terminal' not in top and 'utilities' not in top and any(k in body for k in PICKER_BODY): screen.save('picker') @@ -293,9 +379,53 @@ def run_install(args, proc, qmp, log): screen.stable_since = time.time() continue - # once the bootstrap command is typed, only the picker (above) matters - if typed_at is not None: + # After the install, the loginwindow/desktop is a BRIGHT gray screen, unlike + # the dark install/boot screens (Apple logo). The vmix agent powers the VM + # off if it runs (cron/daemon); if BTM blocks it, we power down here so the + # build still completes with a bootable, installed image. Brightness is a + # far more reliable signal than OCR of the faint "password" text. + bright = screen.mean() > 80 + loginish = (typed_at is not None and bright and 'terminal' not in top + and 'utilities' not in top and not any(k in body for k in PICKER_BODY)) + if loginish: + if login_since is None: + login_since = now + log('bright post-install screen (loginwindow/desktop) — OS installed; grace before powerdown') + elif now - login_since > args.login_grace: + screen.save('loginwindow') + log(f'loginwindow persisted {now - login_since:.0f}s, powering down (install complete)') + try: + qmp.system_powerdown() + except Exception as e: # noqa: BLE001 + log(f'powerdown failed: {e}') + for _ in range(90): + if proc.poll() is not None: + return 0 + time.sleep(1) + proc.kill() + return 0 continue + else: + login_since = None + + # once the bootstrap command is typed, only the picker (above) and an + # unexpected return to Recovery matter (post-prepare reboot landed on the + # recovery instead of the installer — restart the install then). + if typed_at is not None: + if ('utilities' in top or 'recovery' in top): + if recovery_start is None: + recovery_start = now + if now - typed_at > 120 and now - recovery_start > 45: + log('unexpectedly back at Recovery after install started — restarting install') + typed_at = None + terminal_attempts = 0 + recovery_start = None + # fall through to the recovery/terminal handling below + else: + continue + else: + recovery_start = None + continue if 'terminal' in top: screen.save('terminal') @@ -309,6 +439,7 @@ def run_install(args, proc, qmp, log): screen.save('recovery') terminal_attempts += 1 log(f'recovery window (attempt {terminal_attempts}), opening Terminal') + last_term_action = now open_terminal(qmp, log) if terminal_attempts >= 3: time.sleep(8) @@ -339,30 +470,95 @@ def run_install(args, proc, qmp, log): def run_boot(args, proc, qmp, log): + """Wait for the VM to power itself off (customize/generalize/first-boot), + handling the OpenCore picker and kicking a hung boot with a system reset.""" + PICKER_BODY = ('base system', 'macos installer', 'macintosh hd', 'rel-1', 'rel-0') screen = Screen(qmp, args.debug_dir, log) start = time.time() last_periodic = 0 + resets = 0 + blank_since = None + login_since = None while True: rc = proc.poll() if rc is not None: return rc if time.time() - start > args.timeout: try: - screen.grab() - screen.save('timeout') + screen.grab(); screen.save('timeout') except Exception: # noqa: BLE001 pass log('timeout reached, killing QEMU') proc.kill() return 124 time.sleep(args.interval) - if time.time() - last_periodic > args.periodic: - last_periodic = time.time() + try: + screen.grab() + except Exception as e: # noqa: BLE001 + log(f'screendump failed ({e})') + time.sleep(2) + continue + now = time.time() + if now - last_periodic > args.periodic: + last_periodic = now + screen.save('periodic') + if now - start < args.min_boot or screen.stable_for() < args.settle: + continue + if screen.is_blank(): + if blank_since is None: + blank_since = now + elif now - blank_since > args.halt_timeout and disk_idle(args): + screen.save('halt') + log(f'guest halted (black {now - blank_since:.0f}s, disk idle); killing QEMU') + proc.kill() + try: + proc.wait(timeout=10) + except Exception: # noqa: BLE001 + pass + return 0 + continue + blank_since = None + top = screen.menubar_text() + body = screen.ocr() + if 'terminal' not in top and 'utilities' not in top and any(k in body for k in PICKER_BODY): + screen.save('picker') + log('OpenCore boot picker, pressing Return') + qmp.send_key('ret') + screen.stable_since = time.time() + login_since = None + continue + # bright post-boot screen (loginwindow/desktop) => booted; power down if the + # agent did not (so customize/generalize completes even if BTM blocks it) + if screen.mean() > 80 and 'terminal' not in top and 'utilities' not in top: + if login_since is None: + login_since = now + log('bright screen (loginwindow/desktop) after boot; grace before powerdown') + elif now - login_since > args.login_grace: + screen.save('loginwindow') + log(f'loginwindow persisted {now - login_since:.0f}s, powering down') + try: + qmp.system_powerdown() + except Exception as e: # noqa: BLE001 + log(f'powerdown failed: {e}') + for _ in range(90): + if proc.poll() is not None: + return 0 + time.sleep(1) + proc.kill() + return 0 + continue + else: + login_since = None + if 'terminal' not in top and 'utilities' not in top and screen.mean() < 40 \ + and screen.stable_for() > args.stall_reset and disk_idle(args) and resets < args.max_resets: + resets += 1 + screen.save('stall-reset') + log(f'boot hung ({screen.stable_for():.0f}s frozen, dark, disk idle), system_reset #{resets}') try: - screen.grab() - screen.save('periodic') + qmp.system_reset() except Exception as e: # noqa: BLE001 - log(f'screendump failed ({e})') + log(f'system_reset failed: {e}') + screen.stable_since = time.time() def main(): @@ -376,6 +572,12 @@ def main(): p.add_argument('--settle', type=float, default=12.0, help='seconds a screen must be unchanged to act on it') p.add_argument('--min-boot', type=float, default=45.0, help='seconds before the first action') p.add_argument('--max-actions', type=int, default=8) + p.add_argument('--stall-reset', type=float, default=360.0, help='reset the VM if a non-Terminal screen is frozen this long (boot hang)') + p.add_argument('--max-resets', type=int, default=6) + p.add_argument('--halt-timeout', type=float, default=150.0, help='after the bootstrap, a pure-black screen this long means the guest halted (macOS shutdown does not ACPI-power-off QEMU)') + p.add_argument('--progress-file', default=None, help='a file (the system disk) whose mtime shows guest activity; resets/halt only fire when it is also idle, so a slow-but-working boot is never interrupted') + p.add_argument('--disk-idle', type=float, default=90.0, help='seconds of no writes to --progress-file that count as idle') + p.add_argument('--login-grace', type=float, default=240.0, help='seconds to wait at the loginwindow for the agent to power off before the driver powers down itself') p.add_argument('--command', default='diskutil mount VMIX;sh /Volumes/VMIX/run.sh') p.add_argument('qemu', nargs=argparse.REMAINDER) args = p.parse_args() diff --git a/lib/images/macos/tahoe/default.nix b/lib/images/macos/tahoe/default.nix index cd9fa7c..7bb6d48 100644 --- a/lib/images/macos/tahoe/default.nix +++ b/lib/images/macos/tahoe/default.nix @@ -3,6 +3,12 @@ let up = macos.upstream.tahoe; # 18 GB full installer (App Store InstallAssistant.pkg, pinned) installer = pkgs.fetchurl { inherit (up.installer) url hash; name = "InstallAssistant.pkg"; }; - recovery = macos.fetchRecovery { inherit (up.recovery) shortname sha256; }; + # Recovery BaseSystem.dmg: a pre-verified local store file when `recovery.file` is set + # (Apple's CDN rotates Sequoia/Tahoe during the rollout), else fetched + pinned. + # `nix store add-path --name macos-tahoe-BaseSystem.dmg BaseSystem.dmg` yields the same + # content-addressed path on any host that has the file. + recovery = if up.recovery ? file + then builtins.storePath up.recovery.file + else macos.fetchRecovery { inherit (up.recovery) shortname sha256; }; in import ./images.nix { inherit pkgs lib system macos installer recovery; } diff --git a/lib/images/macos/upstream.json b/lib/images/macos/upstream.json index a67c3bd..7730b94 100644 --- a/lib/images/macos/upstream.json +++ b/lib/images/macos/upstream.json @@ -9,7 +9,10 @@ }, "recovery": { "shortname": "tahoe", - "sha256": "edddd0d5869caaa12e29e6996a04f11590280580976a119dbd42c24fa62fe18e" + "sha256": "edddd0d5869caaa12e29e6996a04f11590280580976a119dbd42c24fa62fe18e", + "file": "/nix/store/fqpih1dmg826cfxcyyxn4qm08pvcxgz4-macos-tahoe-BaseSystem.dmg", + "version": "26.6.2", + "build": "25G83" } }, "opencore": {