Minishift notes

Background: attempting to allow external hosts on the same network as the KVM host on which minishift VM is running to be able to access the minishift web UI and optionally via SSH. In this example, is the IP assigned to minishift after starting it, and is the IP range of the network on which the KVM host is running.

On my KVM Linux host, installing libvirtd had created the virbr0 bridge. Subsequently setting up minishift setup a second bridge named virbr1 to which the vNIC assigned to the minishift VM is slaved. Interestingly enough, the minishift VM is then given 2x interfaces with one attached to each bridge (virbr0 and virbr1). In the minishift VM, the default route is assigned to the interface (e.g., eth0) attached to the first bridge (virbr0) with IP, even though when minishift starts, it displays a URL with the IP address ( assigned to the interface connected to the second bridge as the way to access the minishift web UI. This works fine as long as you are attempting to access the URL from the KVM host, but won’t work without some extra steps if you want to access the minishift UI from an external host.
wlp5s0 is the “public” interface on my KVM host.

Setup and start minishift (equivalent to RedHat CDK if you have the right subscription):
itababa@itamint:~$ sudo apt update -y
itababa@itamint:~$ sudo apt install qemu-kvm qemu-system qemu-utils python3 python3-pip libvirt-clients libvirt-daemon-system bridge-utils virtinst libvirt-daemon virt-manager cpu-checker -y
itababa@itamint:~$ usermod -aG libvirt $(whoami)
itababa@itamint:~$ newgrp libvirt
itababa@itamint:~$ sudo systemctl enable libvirtd
itababa@itamint:~$ sudo systemctl start libvirtd
itababa@itamint:~$ sudo virsh net-start default
itababa@itamint:~$ sudo virsh net-autostart default
itababa@itamint:~$ sudo curl -L -o /usr/local/bin/docker-machine-driver-kvm
itababa@itamint:~$ sudo chmod +x /usr/local/bin/docker-machine-driver-kvm
itababa@itamint:~$ curl -L -o minishift-1.34.3-linux-amd64.tgz
itababa@itamint:~$ tar xf minishift-1.34.3-linux-amd64.tgz
itababa@itamint:~$ sudo cp minishift-1.34.3-linux-amd64/minishift /usr/local/bin/
itababa@itamint:~$ minishift start

NOTE: the “minishift start” command will display the minishift web UI at the end of its run.

Settings needed on the KVM host:
- configure the KVM host to redirect connections to it on 8443/tcp to the minishift host ( Optionally redirect 2222/tcp to the minishift host as well:
iptables -F
iptables -F -t nat
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/ipv4/conf/virbr0/proxy_arp
echo 1 > /proc/sys/net/ipv4/conf/virbr1/proxy_arp    (required so that the host can answer when the minishift VM tries to find the ARP of external hosts, otherwise the minshift will not respond to any connection attempts)
iptables -t nat -A POSTROUTING -o wlp5s0 -j MASQUERADE
iptables -t nat -A PREROUTING -i wlp5s0 -p tcp --dport 8443 -j DNAT --to-destination
iptables -t nat -A PREROUTING -i wlp5s0 -p tcp --dport 2222 -j DNAT --to-destination
Settings needed on the minishift VM after it is running (execute "minishift ssh" to SSH into the minishift VM):
- NOTE: this whole section can be skipped if you choose to "DNAT" port 8443/tcp to the IP address on the virbr0 vNIC on the KVM host:
itababa@itamint:~$ minishift ssh
[docker@minishift ~]$
 sudo su -
[root@minishift ~]# ip a     (find the interface with the 192.168.42.x IP e.g., eth1)
[root@minishift ~]# ip route add via dev eth1 (this is because the default route uses the NIC with 192.168.122.x IP and used by minishift to access the Internet)
[root@minishift ~]# ip route show
default via dev eth0 proto dhcp metric 100 dev docker0 proto kernel scope link src dev eth1 scope link dev eth1 proto kernel scope link src metric 101 dev eth0 proto kernel scope link src metric 100
[root@minishift ~]#
NOTE: if you restart the minishift VM, you have to SSH into it again and re-add the route to the network.

Settings needed on the external host:
In this example, I want to access the minishift web UI from a Windows system on my home network. You need to add a route to the minishift IP address using the KVM host IP as the gateway (using the command line):
C:\Windows\system32>route add MASK

Launch a browser on the external host and go directly to

NOTE: if you attempt to login via , you will encounter this error/bug after entering your credentials (e.g., admin/admin)
Error: “Error. Invalid request. Client state could not be verified. Return to the console.”
Bug: “”
Found the solution (use the /console/catalog URL):

after login with admin/admin credential

Other commands:
minishift delete –clear-cache (solution for error similar to: “Cannot start – error starting the VM: error getting the state for host: machine dies not exist-docker” when attempting to start minishift)
sudo virsh stop minishift; sudo undefine minishift; (might be needed if the “delete” command does not fix the issue)
sudo arp -a (on the KVM host to see the IPs of the minishift VM or SSH into the minishift VM)

– To restart the minishift VM (from the KVM host):
itababa@itamint:~$ minishift stop
itababa@itamint:~$ minishift start


NOTE: for educational purposes only

GUIDE: Used OSX-KVM with modifications:

  • I installed the latest QEMU (compiled from source):
    root@itamint:~/# apt install -y gcc make ninja-build libglib2.0-dev libpixman-1-dev ncurses-dev libusb-dev libusb-1.0-0-dev libusbredirparser1 libusbredirhost1 usbutils
    root@itamint:~# git clone
    root@itamint:~#cd qemu
    root@itamint:~/qemu# ./configure –enable-vde –enable-libusb –prefix=/usr/local –target-list=”i386-softmmu x86_64-softmmu”
    root@itamint:~/qemu# make
    root@itamint:~/qemu# make install
    root@itamint:~/qemu# qemu-system-x86_64 –version
    QEMU emulator version 7.2.50 (v7.2.0-2313-g9832009d9d)

  • Install libvirtd/QEMU
    root@itamint:~# apt update -y
    root@itamint:~# apt install qemu-kvm qemu-system qemu-utils python3 python3-pip libvirt-clients libvirt-daemon-system bridge-utils virtinst libvirt-daemon virt-manager cpu-checker -y
    root@itamint:~# usermod -aG root
    root@itamint:~# newgrp kvm
    root@itamint:~# systemctl enable libvirtd
    root@itamint:~# systemctl start libvirtd
    root@itamint:~# virsh net-start default
    root@itamint:~# virsh net-autostart default

    NOTE: this is to make networking config easy. We won’t use the qemu version 6.x installed above. We will use the much newer version we compiled earlier which we installed in /usr/local/bin/ whereas the one installed by the apt command puts qemu in /usr/bin/. To confirm you are using the right qemu, run “which qemu-system-x86_64”

  • Setup networking so that the VM has network access:
    I have put all the networking and firewall rules in a script which can then be executed with a command such as “sudo /home/itababa/”

    itababa@itamint:~/OSX-KVM$ cat /home/itababa/

    # create a VDE switch and add a subnet range to it. also add an IP address to the switch as it will be the gateway for connected VMs:
    vde_switch -tap vde0 -daemon
    ip link set dev vde0 up
    ip addr add dev vde0
    ip route add dev vde0
    echo 1 > /proc/sys/net/ipv4/conf/vde0/proxy_arp

    # NOTE: it is possible to setup DHCP on the switch but I will be using static IPs in this guide

    # Internet access to the VMs (execute on the QEMU hypervisor host) where wlp5s0 is the WAN NIC on the Ubuntu QEMU host:
    echo 1 > /proc/sys/net/ipv4/ip_forward
    iptables -t nat -A POSTROUTING -o wlp5s0 -j MASQUERADE
    iptables -I FORWARD 1 -i vde0 -j ACCEPT
    iptables -I FORWARD 1 -o vde0 -m state –state RELATED,ESTABLISHED -j ACCEPT

    # On the QEMU hypervisor (Linux) host, configure the rules for the ports to be forwarded to the OSX VM:
    iptables -A INPUT -p tcp –dport 2222 -j ACCEPT
    iptables -t nat -A PREROUTING -p tcp –dport 2222 -j DNAT –to-destination
    iptables -A INPUT -p tcp –dport 5905 -j ACCEPT
    iptables -t nat -A PREROUTING -p tcp –dport 5905 -j DNAT –to-destination
    # end of networking script

  • “Install” OSX in the VM using OSX-KVM:

itababa@itamint:~/$ sudo echo 1 | sudo tee /sys/module/kvm/parameters/ignore_msrs
itababa@itamint:~/$ sudo cp kvm.conf /etc/modprobe.d/kvm.conf

itababa@itamint:~/$ sudo groupadd kvm
itababa@itamint:~/$ sudo groupadd libvirt
itababa@itamint:~/$ sudo groupadd input

itababa@itamint:~/$ sudo usermod -aG kvm $(whoami)
itababa@itamint:~/$ sudo usermod -aG libvirt $(whoami)
itababa@itamint:~/$ sudo usermod -aG input $(whoami)

itababa@itamint:~/$ cd ~
itababa@itamint:~/$ git clone –depth 1 –recursive
itababa@itamint:~/$ cd OSX-KVM
itababa@itamint:~/OSX-KVM$ pwd

  • Fetch the Ventura Installer (option 6):
    itababa@itamint:~/OSX-KVM$ ./

  • Convert the downloaded BaseSystem.dmg file into the BaseSystem.img file.
    itababa@itamint:~/OSX-KVM$ dmg2img -i BaseSystem.dmg BaseSystem.img

  • Create a virtual HDD image where macOS will be installed.
    itababa@itamint:~/OSX-KVM$ qemu-img create -f qcow2 mac_hdd_ng.img 128G

  • Edit the script and make the following changes:
    – increase the RAM from 4096MB to 8192MB (ALLOCATED_RAM variable)
    – add avx2 to the CPU flags list (MY_OPTIONS variable)
    – change the CPU from Penryn to Cascadelake-Server-noTSX (only one that didn’t cause random freezing after installation and reboot loop during installation from the several I tested. Unfortunately the VM still freezes/hangs consistently between 10 and 12 minutes after startup.)
    – since we are VDE, comment out the default NIC entry starting with “-netdev user” and replace with our VDE line (see below).
    – Below is how the relevant lines looked in my modified file:
    root@itamint:/home/itababa/OSX-KVM# diff
    ALLOCATED_RAM=”8192″ # MiB
    -enable-kvm -m “$ALLOCATED_RAM” -cpu Cascadelake-Server-noTSX,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,”$MY_OPTIONS”
    -device virtio-net-pci,netdev=net0,mac=52:54:00:e6:5d:16 -netdev vde,id=net0
  • Run the script (with admin rights as root or using sudo) to create the VM and start the installation (this starts a VNC session listening on localhost:5901 for example. Use Mobaxterm/VNC or putty tunnel+VNC client to access the VNC session)
    itababa@itamint:~/OSX-KVM$ sudo ./
    VNC server running on (output of executing the script)
    NOTE: without “sudo” or “root” user, you will get an error similar to “Could not open vde: No such file or directory”

  • Connnect to the VNC session (using a VNC client):

  • Go through the installation steps (use Disk Utility to erase the target disk, then install Ventura)

  • Optional: Login into the OSX and enabled “remote management” (VNC) and “remote login” (SSH) (from “System Settings …” – it is no longer called “System Preferences”)

  • Optional: if you enabled “remote management” (VNC) and/or “remote login” (SSH), you can connect directly to the OSX VM via a VNC client (using the IP address of the Linux host and port 5901) and/or via SSH (using the IP address of the Linux host and port 2222). The connections work because we included the relevant forwarding rules in the firewall script (above).

  • Naviate to “System Settings” > Network > Ethernet > “Details…”. Assign a static IP address such as (see the script above) to the OSX with subnet mask and Router (IP address of the VDE switch). Also assign the VDE switch IP address as the DNS Server.

  • Optional: Update and comment out the 2 lines that attaches the installer disk to the VM i.e.,
    # -device ide-hd,bus=sata.3,drive=InstallMedia
    # -drive id=InstallMedia,if=none,file=”$REPO_PATH/BaseSystem.img”,format=raw
  • Optional: shutdown OSX. Modify so that it starts the VM “headless” (i.e., running as a background process):
    Comment out the line “-monitor stdio” and add a line below it with “-daemonize” without the double-quotes.

  • Start the VM in the (OSX-KVM directory) with the command: sudo ./

  • Configure the VM to auto-boots from the OSX disk, run the command at the UEFI shell:
    bcfg boot add 0 FS0:\EFI\boot\BOOT_X64.efi “my_boot”

  • Boot to OSX from the UEFI shell with the command sequence (ENTER key after each one): FS0: > cd EFI > cd BOOT > BOOTx64.efi

  • You can now connect to the OSX VM with VNC (Linux host IP and port 5905 in this example) or SSH (Linux host IP and port 2222 in this example).

NOTE: Several methods to attach a USB stick/device (e.g., one containing TimeMachine backups) to a QEMU VM

Option 1: In Linux Mint (Debian/Ubuntu), attached USB storage gets the group “disk”. So add your normal user to that group.
$ sudo usermod -aG disk itababa
$ newgrp disk

– Then add the device nodename of the USB device to the “args” list for QEMU in the file (get the device file from the output of “blkid” or “lsblk” (e.g., /dev/sdb)
-hdc /dev/sdb

– you can also use the following 2x lines (equivalent to the single “-hdc /dev/sdb” line above)
-drive id=USBstick,if=none,file=”/dev/sdb”
-device ide-hd,bus=sata.5,drive=USBstick

Note: in OSX, since the USB is “direct-attached”, it is treated as a HDD. To make the volume appear on the desktop, go to “Finder” (menu) > “Settings”. Check/select/enable the “Hard disks” option.

Option 2: Add the USB device’s “location” to the args list for QEMU in the file (get the hostbus and hostaddr from the output of the “lsusb” commmand). You must also change the permission on the device files (e.g., “sudo chmod o+rw /dev/bus/usb/002/*”). This operation does not survive a reboot of the physical host or re-insertion of the USB, so if you want it permanent consider UDEV rules):
-device qemu-xhci,id=usbxhci
-usb -device usb-host,hostbus=2,hostaddr=6

For example, in the “lsusb” output below, “Bus” is “hostbus” and “Device” is “hostaddr”
root@itamint:/home/itababa/OSX-KVM# lsusb

Bus 002 Device 006: ID 05dc:a838 Lexar Media, Inc. JumpDrive Tough

IMPORTANT NOTE: to prevent the OSX on QEMU from freezing/hanging (after about 10 mins uptime):
– Go to “System Settings …” > Energy Saver > Conserve battery > “Put hard disks to sleep when possible” disabled.
– Go to “System Settings …” > “Lock Screen” > “Turn display off when inactive” to “Never”
– Also, you MUST login to an account, if you leave the system on the login screen, it will freeze/hang.

* If you restore a time machine back along with “System & Networking”, check afterwards that the two settings above have not reverted to their defaults.

  • NOTE: Take snapshots often (e.g., before using “Migration Assistant” to restore TimeMachine backups from another OSX)
    – take an internal snapshot: qemu-img snapshot -c mac_hdd_ng.img
    – list internal snapshots: qemu-img snapshot -l mac_hdd_ng.img
    – revert/restore to a snapshot: qemu-img snapshot -a mac_hdd_ng.img
  • NOTE: IF “managed device attestation” is configured on your source OSX from which you restored a Time Machine backup, you may notice ACME (or a similar policy enforcement) app is installed and running.
    IF your source OSX is part of a corporate network, unless you decide to connect to the corporate VPN (unlikely for the purpose of this procedure), ACME won’t be able to validate your OSX VM config and may isolate your restored OSX VM (no networking access). To fix this, temporarily stop the ACME app, or permanently remove it:
    – Use the Utilities > “Activity Monitor” > to stop the ACME process
    – Go to Applications folder and delete (“move to trash”) the ACME app shortcut
    – Go to folder ~/Library/ and delete (“move to trash”) the ACME folder

References: (enable remote management i.e., VNC)