housekeeping

This commit is contained in:
bill-auger 2019-12-21 06:57:35 -05:00
parent 4e9bf45e05
commit ca4cb2eb44
4 changed files with 140 additions and 103 deletions

82
README

@ -5,70 +5,104 @@ parabola-vmbootstrap
This is a collection of scripts for creating and booting parabola virtual
machine images for use with qemu.
------------------------------
virtual machine image creation
------------------------------
To create a new virtual machine image, run
$> ./pvmbootstrap.sh [path to image] [arch]
$> pvmbootstrap [options] <path-to-image> <arch>
where arch is one of the supported parabola arches, which are currently:
... where <arch> is one of the parabola arches, which are currently:
official: [ i686, x86_64, armv7h ]
unofficial: [ ppc64le, riscv64 ]
supported: [ armv7h, i686, x86_64 ]
experimental: [ ppc64le, riscv64 ]
the script will attempt to bootstrap a virtual machine of the selected
archituecture in the output file specified. If the output file already exists,
The script will attempt to bootstrap a virtual machine of the selected
archituecture to the output file specified. If the output file already exists,
the script will emit a warning and ask for confirmation to proceed.
the creation can be influenced by providing one or more of the following
arguments to pvmbootstrap.sh:
The creation can be influenced by providing one or more of the following
options to pvmbootstrap:
-s size -- set the size of the created VM image (default: 64G)
-M mirror -- set the mirror to fetch packages from
(default: https://repo.parabola.nu/$repo/os/$arch)
-H hook -- add a confiuration hook to be executed. this can be the path to
a shell script or one of the existing hooks described below
-H <hook> -- Enable a hook to customize the created image. This can be
the path to a custom script, or one of the predefined hooks
described below. The VM will boot the newly created image,
and the hook script(s) will be executed once each within
the running VM. This option can be specified multiple times.
-k <kernel> -- Choose an additional kernel package (default: linux-libre).
This option can be specified multiple times; but note that
'linux-libre' will be installed, regardless of this option.
-M <mirror> -- Set the mirror from which to fetch packages
(default: https://repo.parabola.nu/$repo/os/$arch)
-O -- Bootstrap an openrc system instead of a systemd one
-p <package> -- Specify additional packages to be installed in the VM image.
This option can be specified multiple times.
-s <img_size> -- Set the size (in GB) of the VM image (minimum: 1, default: 64)
-S <swap_size> -- Set the size (in MB) of the swap partition (default: 0)
The creation hooks currently supported are:
ethernet-dhcp:
'ethernet-dhcp':
this hook will setup ethernet in the VM by enabling systemd-resolved and
This hook will setup ethernet in the VM by enabling systemd-resolved and
openresolv properly, as well as creating and enabling a systemd-networkd
configuration.
--------------------
virtual machine boot
--------------------
To boot a created virtual machine, run:
$> ./pvmboot.sh [path to image] [additional qemu args...]
$> pvmboot [options] <path_to_image> [qemu-args ...]
the script will attempt to determine the architecture of the provided virtual
The script will attempt to determine the architecture of the provided virtual
machine image, and set the qemu executable and sane default flags for the qemu
invocation automatically, including kvm acceleration, if available for the
target architecture.
additionally, the script will evaluate the DISPLAY environment variable to
Additionally, the script will evaluate the DISPLAY environment variable to
determine whether a graphical desktop environment is available, and will start
the image in serial console mode if necessary. This behavior can be forced by
unsetting DISPLAY before executing the script:
$> DISPLAY= ./pvmboot.sh [...]
$> DISPLAY= pvmboot ./an.img
The default flags can be overwritten or extended, for example to allocate more
memory to the machine, by specifying additional qemu parameters on the command
The default kernel installed by pvmbootstrap is 'linux-libre'.
The -k option ca be used to specify an alternate kernel to boot.
$> pvmboot -k linux-libre-lts ./an.img
When the VM will boot into graphical mode, the serial console can be redirected
to the host console by passing the -r option.
$> pvmboot -r ./an.img
The qemu launch configuration can be customized, for example to allocate more
memory to the machine, by specifying additional qemu options on the command
line following the virtual machine image name:
$> DISPLAY= ./pvmboot [path to image] -m 2G
$> DISPLAY= pvmboot ./an.img -m 2G
------------------------
release tarball creation
------------------------
To convert a virtual machine image into a parabola release tarball, run:
$> ./pvm2tarball.sh [path to image]
$> pvm2tarball <path_to_image>
This will attempt to mount the rootfs and the /boot partition, and tar the
contents to an output file, filtering out all unneeded files.
contents to an output file, filtering out all unneeded files. The /boot
partition is optional if the kernel and initramfs are in /boot; though
pvmbootstrap always creates a /boot partition.
for example, to generate the parabola armv7h release tarball:
$> img_filename=parabola-systemd-cli-armv7h-tarball-$(date +%Y.%m).img
$> pvmbootstrap -s 1 -H ethernet-dhcp $img_filename armv7h
$> pvm2tarball $img_filename

@ -3,6 +3,7 @@
# parabola-vmbootstrap -- create and start parabola virtual machines #
# #
# Copyright (C) 2017 - 2019 Andreas Grapentin #
# Copyright (C) 2019 - 2020 bill-auger #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
@ -22,20 +23,22 @@
source "$(librelib messages)"
usage() {
print "USAGE: %s [-h] [-o FILE] IMG" "${0##*/}"
prose "Produce a parabola release tarball from IMG."
print "USAGE:"
print " pvm2tarball [-h] [-o <FILENAME>] <IMG>"
echo
prose "IMG is expected to be a valid parabola image, ideally freshly bootstrapped
using pvmbootstrap. If FILE is not specifed, generate an archive name
from IMG and place it in the current working directory"
prose "Produce a parabola release tarball from <IMG>."
echo
prose "<IMG> is expected to be a valid parabola image, ideally freshly bootstrapped
using pvmbootstrap. If <FILENAME> is not specifed, generate an archive name
from <IMG> and place it in the current working directory"
echo
echo "Supported options:"
echo " -o FILE Write the generated tar archive to FILE instead of"
echo " generating a name for the archive from IMG"
echo " -h Display this help and exit"
echo " -o <FILENAME> Write the generated tar archive to FILE instead of"
echo " generating a name for the archive from IMG"
echo " -h Display this help and exit"
echo
echo "This script is part of parabola-vmbootstrap. source code available at:"
echo " <https://git.parabola.nu/~oaken-source/parabola-vmbootstrap.git>"
echo " <https://git.parabola.nu/parabola-vmbootstrap.git>"
}
pvm_mount() {

@ -3,6 +3,7 @@
# parabola-vmbootstrap -- create and start parabola virtual machines #
# #
# Copyright (C) 2017 - 2019 Andreas Grapentin #
# Copyright (C) 2019 - 2020 bill-auger #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
@ -29,7 +30,9 @@ RedirectSerial=0
usage() {
print "USAGE: %s [-h] [-k <kernel>] [-r] <img> [qemu-args ...]" "${0##*/}"
print "USAGE:"
print " pvmboot [-h] [-k <kernel>] [-r] <img> [qemu-args ...]"
echo
prose "Determine the architecture of <img> and boot it using qemu. <img> is assumed
to be a valid, raw-formatted parabola virtual machine image, ideally
created using pvmbootstrap. The started instances are assigned
@ -57,63 +60,63 @@ usage() {
echo " -r Redirect serial console to host console, even in graphics mode"
echo
echo "This script is part of parabola-vmbootstrap. source code available at:"
echo " <https://git.parabola.nu/~oaken-source/parabola-vmbootstrap.git>"
echo " <https://git.parabola.nu/parabola-vmbootstrap.git>"
}
pvm_mount() {
if ! file "$1" | grep -q ' DOS/MBR '; then
error "%s: does not seem to be a raw qemu image." "$1"
if ! file "$imagefile" | grep -q ' DOS/MBR '; then
error "%s: does not seem to be a raw qemu image." "$imagefile"
return "$EXIT_FAILURE"
fi
msg "mounting filesystems"
trap 'pvm_umount' INT TERM EXIT
workdir="$(mktemp -d -t pvm-XXXXXXXXXX)" || return "$EXIT_FAILURE"
loopdev="$(sudo losetup -fLP --show "$1")" || return "$EXIT_FAILURE"
sudo mount "$loopdev"p1 "$workdir" \
|| sudo mount "$loopdev"p2 "$workdir" || return "$EXIT_FAILURE"
workdir="$(mktemp -d -t pvm-XXXXXXXXXX)" || return "$EXIT_FAILURE"
loopdev="$(sudo losetup -fLP --show "$imagefile")" || return "$EXIT_FAILURE"
sudo mount "$loopdev"p1 "$workdir" || \
sudo mount "$loopdev"p2 "$workdir" || return "$EXIT_FAILURE"
}
pvm_umount() {
msg "un-mounting filesystems"
trap - INT TERM EXIT
[ -n "$workdir" ] && (sudo umount "$workdir"; rmdir "$workdir")
unset workdir
[ -n "$loopdev" ] && sudo losetup -d "$loopdev"
unset workdir
unset loopdev
}
pvm_probe_arch() {
local kernel
kernel=$(find "$workdir" -maxdepth 1 -type f -iname '*vmlinu*' | head -n1)
if [ -z "$kernel" ]; then
warning "%s: unable to find kernel binary" "$1"
warning "%s: unable to find kernel binary" "$imagefile"
return "$EXIT_FAILURE"
fi
# attempt to get kernel arch from elf header
arch="$(readelf -h "$kernel" 2>/dev/null | grep Machine | awk '{print $2}')"
case "$arch" in
PowerPC64) arch=ppc64; return "$EXIT_SUCCESS";;
RISC-V ) arch=riscv64; return "$EXIT_SUCCESS";;
* ) arch="";;
PowerPC64) arch=ppc64 ; return "$EXIT_SUCCESS" ;;
RISC-V ) arch=riscv64 ; return "$EXIT_SUCCESS" ;;
* ) arch="" ;;
esac
# attempt to get kernel arch from objdump
arch="$(objdump -f "$kernel" 2>/dev/null | grep architecture: | awk '{print $2}' | tr -d ',')"
case "$arch" in
i386 ) arch=i386; return "$EXIT_SUCCESS";;
i386:*) arch=x86_64; return "$EXIT_SUCCESS";;
* ) arch="";;
i386 ) arch=i386 ; return "$EXIT_SUCCESS" ;;
i386:*) arch=x86_64 ; return "$EXIT_SUCCESS" ;;
* ) arch="" ;;
esac
# attempt to get kernel arch from file magic
arch="$(file "$kernel")"
case "$arch" in
*"ARM boot executable"*) arch=arm; return "$EXIT_SUCCESS";;
* ) arch="";;
*"ARM boot executable"*) arch=arm ; return "$EXIT_SUCCESS" ;;
* ) arch="" ;;
esac
# no more ideas; giving up.
@ -121,10 +124,11 @@ pvm_probe_arch() {
pvm_native_arch() {
local arch
case "$1" in
arm*) arch=armv7l;;
*) arch="$1";;
esac
case "$1" in
arm*) arch=armv7l ;;
* ) arch="$1" ;;
esac
setarch "$arch" /bin/true 2>/dev/null || return
}
@ -136,25 +140,25 @@ pvm_guess_qemu_args() {
fi
# if we're running a supported arch, enable kvm
if pvm_native_arch "$2"; then qemu_args+=(-enable-kvm); fi
if pvm_native_arch "$arch"; then qemu_args+=(-enable-kvm); fi
# find root filesystem partition (necessary for arches without bootloader)
local root_loopdev_n=$(echo $(parted "$1" print 2> /dev/null | grep ext4) | cut -d ' ' -f 1)
local root_loopdev_n=$(echo $(parted "$imagefile" print 2> /dev/null | grep ext4) | cut -d ' ' -f 1)
local root_loopdev="$loopdev"p$root_loopdev_n
local root_vdev=/dev/vda$root_loopdev_n
if [[ -b "$root_loopdev" ]]
then
msg "found root filesystem loop device: %s" "$root_loopdev"
else
error "%s: unable to determine root filesystem loop device" "$1"
error "%s: unable to determine root filesystem loop device" "$imagefile"
return "$EXIT_FAILURE"
fi
# set arch-specific args
local kernel_console
case "$2" in
case "$arch" in
i386|x86_64|ppc64)
qemu_args+=(-m $DEF_RAM_MB -hda "$1")
qemu_args+=(-m $DEF_RAM_MB -hda "$imagefile")
# unmount the unneeded virtual drive early
pvm_umount ;;
arm)
@ -164,7 +168,7 @@ pvm_guess_qemu_args() {
-kernel "$workdir"/vmlinuz-${Kernel}
-initrd "$workdir"/initramfs-${Kernel}.img
-append "${kernel_console}rw root=${root_vdev}"
-drive "if=none,file=$1,format=raw,id=hd"
-drive "if=none,file=${imagefile},format=raw,id=hd"
-device "virtio-blk-device,drive=hd"
-netdev "user,id=mynet"
-device "virtio-net-device,netdev=mynet") ;;
@ -181,7 +185,7 @@ pvm_guess_qemu_args() {
-netdev "user,id=usernet"
-device "virtio-net-device,netdev=usernet") ;;
*)
error "%s: unable to determine default qemu args" "$1"
error "%s: unable to determine default qemu args" "$imagefile"
return "$EXIT_FAILURE" ;;
esac
}
@ -210,24 +214,23 @@ main() {
msg "initializing ...."
local workdir loopdev
pvm_mount "$imagefile" || exit
pvm_mount || exit
local arch
pvm_probe_arch "$imagefile" || exit
pvm_probe_arch || exit
if [ -z "$arch" ]; then
error "image arch is unknown: '%s'" "$arch"
exit "$EXIT_FAILURE"
fi
local qemu_args=()
pvm_guess_qemu_args "$imagefile" "$arch" || exit
pvm_guess_qemu_args || exit
qemu_args+=("$@")
msg "booting ...."
msg "booting VM ...."
(set -x; qemu-system-"$arch" "${qemu_args[@]}")
# clean up the terminal, in case SeaBIOS did something weird
msg "cleaning up ...."
echo -n "[?7h"
pvm_umount
}

@ -3,6 +3,7 @@
# parabola-vmbootstrap -- create and start parabola virtual machines #
# #
# Copyright (C) 2017 - 2019 Andreas Grapentin #
# Copyright (C) 2019 - 2020 bill-auger #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
@ -34,6 +35,7 @@ readonly DEF_SWAP_MB=0
# misc
readonly THIS_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
readonly GUEST_CACHED_PKGS=('ca-certificates-utils')
# options
Hooks=()
@ -85,7 +87,7 @@ usage() {
echo " (systemd only)"
echo
echo "This script is part of parabola-vmbootstrap. source code available at:"
echo " <https://git.parabola.nu/~oaken-source/parabola-vmbootstrap.git>"
echo " <https://git.parabola.nu/parabola-vmbootstrap.git>"
}
pvm_native_arch() {
@ -95,19 +97,19 @@ pvm_native_arch() {
}
pvm_bootstrap() {
msg "starting creation of %s image: %s" "$arch" "$file"
msg "starting creation of %s image: %s" "$arch" "$imagefile"
# create the raw image file
qemu-img create -f raw "$file" "${ImgSizeGb}G" || return "$EXIT_FAILURE"
qemu-img create -f raw "$imagefile" "${ImgSizeGb}G" || return "$EXIT_FAILURE"
# prepare for cleanup
trap 'pvm_cleanup' INT TERM RETURN
# mount the virtual disk
local workdir loopdev
workdir="$(mktemp -d -t pvm-rootfs-XXXXXXXXXX)" || return "$EXIT_FAILURE"
loopdev="$(sudo losetup -fLP --show "$file")" || return "$EXIT_FAILURE"
sudo dd if=/dev/zero of="$loopdev" bs=1M count=8 || return "$EXIT_FAILURE"
workdir="$(mktemp -d -t pvm-rootfs-XXXXXXXXXX)" || return "$EXIT_FAILURE"
loopdev="$(sudo losetup -fLP --show "$imagefile")" || return "$EXIT_FAILURE"
sudo dd if=/dev/zero of="$loopdev" bs=1M count=8 || return "$EXIT_FAILURE"
# partition
local boot_begin="$( [[ "$arch" =~ i686|x86_64 ]] && echo 2 || echo 1 )MiB"
@ -182,8 +184,8 @@ pvm_bootstrap() {
# target arch can't execute natively, pacstrap is going to need help by qemu
local qemu_arch
case "$arch" in
armv7h) qemu_arch=arm ;;
*) qemu_arch="$arch" ;;
armv7h) qemu_arch=arm ;;
* ) qemu_arch="$arch" ;;
esac
local qemu_user_static=$(sudo grep -l -F -e "interpreter /usr/bin/qemu-$qemu_arch-" \
@ -208,30 +210,27 @@ pvm_bootstrap() {
echo -e "[options]\nArchitecture = $arch\n\n" > "$pacconf"
for repo in ${repos[@]}; do echo -e "[$repo]\nServer = $Mirror\n" >> "$pacconf"; done;
# prepare lists of packages
local kernels=(${Kernels[@]})
local pkgs=(${DEF_PKGS[@]} ${Kernels[@]} ${Pkgs[@]})
# prepare package lists
local kernels=( ${Kernels[@]} )
local pkgs=( ${DEF_PKGS[@]} ${Kernels[@]} ${Pkgs[@]} )
local pkgs_cached=( ${GUEST_CACHED_PKGS[@]} )
case "$arch" in
i686|x86_64) pkgs+=(grub) ;;
esac
case "$arch" in
riscv64) ;;
*) pkgs+=("haveged" net-tools) ;;
i686|x86_64) pkgs+=(grub) ;;
riscv64 ) ;;
* ) pkgs+=(haveged net-tools) ;;
esac
(( $IsNonsystemd )) && && pkgs+=(libelogind)
(( ! $IsNonsystemd )) && [[ "${Hooks[@]}" =~ hook-ethernet-dhcp.sh ]] && pkgs+=(dhcpcd)
# remove duplicate package names
# minimize package lists
Kernels=() ; Pkgs=() ;
for kernel in $(printf "%s\n" "${kernels[@]}" | sort -u) ; do Kernels+=($kernel) ; done ;
for pkg in $(printf "%s\n" "${pkgs[@]}" | sort -u) ; do Pkgs+=($pkg) ; done ;
local pkg_guest_cache=(ca-certificates-utils)
# pacstrap! :)
msg "installing packages into the work chroot"
sudo pacstrap -GMc -C "$pacconf" "$workdir" "${pkgs[@]}" || return "$EXIT_FAILURE"
sudo pacstrap -GM -C "$pacconf" "$workdir" "${pkg_guest_cache[@]}" || return "$EXIT_FAILURE"
sudo pacstrap -GMc -C "$pacconf" "$workdir" "${pkgs[@]}" || return "$EXIT_FAILURE"
sudo pacstrap -GM -C "$pacconf" "$workdir" "${pkgs_cached[@]}" || return "$EXIT_FAILURE"
# create an fstab
msg "generating /etc/fstab"
@ -385,13 +384,13 @@ EOF
error "pvmboot not available -- unable to run hooks"
return "$EXIT_FAILURE"
fi
pvmboot_cmd+=("$file" "${qemu_flags[@]}")
pvmboot_cmd+=("$imagefile" "${qemu_flags[@]}")
exec 3>&1
msg "booting the machine to run the pre-init hooks"
DISPLAY='' "${pvmboot_cmd[@]}" | tee /dev/fd/3 | grep -q -F "$hooks_success_msg"
local res=$?
exec 3>&-
! (( $res )) || error "%s: failed to complete preinit hooks" "$file"
! (( $res )) || error "%s: failed to complete preinit hooks" "$imagefile"
return $res
}
@ -443,7 +442,7 @@ main() {
local shiftlen=$(( OPTIND - 1 ))
shift $shiftlen
local file="$1"
local imagefile="$1"
local arch="$2"
local has_params=$( (( $# == 2 )) && echo 1 || echo 0 )
local has_space=$( (( ($ImgSizeGb*1000) >= ($MIN_GB*1000) + $SwapSizeMb )) && echo 1 || echo 0 )
@ -453,32 +452,30 @@ main() {
# determine if the target arch is supported
case "$arch" in
i686|x86_64|armv7h) ;;
ppc64le|riscv64)
warning "arch %s is experimental" "$arch";;
*)
error "arch %s is unsupported" "$arch"
exit "$EXIT_INVALIDARGUMENT";;
i686|x86_64|armv7h) ;;
ppc64le|riscv64 ) warning "arch %s is experimental" "$arch" ;;
* ) error "arch %s is unsupported" "$arch"
exit "$EXIT_INVALIDARGUMENT" ;;
esac
# determine whether the target output file already exists
if [ -e "$file" ]; then
warning "%s: file exists. Continue? [y/N]" "$file"
if [ -e "$imagefile" ]; then
warning "%s: file exists. Continue? [y/N]" "$imagefile"
read -p " " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
exit "$EXIT_FAILURE"
fi
rm -f "$file" || exit
rm -f "$imagefile" || exit
fi
# create the virtual machine
if ! pvm_bootstrap; then
error "bootstrap failed for image: %s" "$file"
error "bootstrap failed for image: %s" "$imagefile"
exit "$EXIT_FAILURE"
fi
msg "bootstrap complete for image: %s" "$file"
msg "bootstrap complete for image: %s" "$imagefile"
}
main "$@"