full rewrite to add more risc-v

This commit is contained in:
Andreas Grapentin 2018-04-24 17:58:58 +02:00
parent 5923111e33
commit 9df4b2cda8
No known key found for this signature in database
GPG Key ID: 7171986E4B745536
13 changed files with 522 additions and 721 deletions

88
README

@ -1,88 +1,34 @@
parabola-arm-imagebuilder
=========================
parabola-imagebuilder
=====================
This is a collection of scripts creating parabola images for use with qemu.
image creation
--------------
this is a collection of scripts creating parabola arm images for use with qemu
with the original goal of building parabola arm packages on these machines.
Development focus has since shifted towards creating parabola-arm release
tarballs.
to create a new virtual machine image, run
To create a new virtual machine image, run
$> sudo ./create.sh
by default, the creation script will use a ParabolaArm release tarball to
create the virtual machine. Alternatively, to install an archlinuxarm tarball
and migrating the installed system to parabola in-place, set the environment
variable ARCHBOOTSTRAP to 1:
The creation is influenced by the following environment variables:
$> sudo ARCHBOOTSTRAP=1 ./create.sh
ARCH - the target architecture of the image. default: armv7h
Optionally, to create a new virtual machine with a packaging environment, set
the environment variable DEVSETUP to 1:
SIZE - the size of the root image. default: 64GiB
$> sudo DEVSETUP=1 ./create.sh
MIRROR - the mirror used to pacstrap the image, anything valid in a `Server =`
line can go here.
default: https://redirector.parabola.nu/\$repo/os/\$arch}
The packaging environment setup script makes use of several files and packages
present on already setup parabola development machines. if your setup is
different, you might have to modify src/stage3.sh accordingly.
The created images are stored in the build/ directory.
Places the scripts check the host machine for configuration files are:
/etc/makepkg.conf - for PACKAGER and GPGKEY
~/.gnupg ~/.ssh ~/.gitconfig - copied verbatim to the VM
The scripts assume that the following programs are available and in $PATH:
qemu-img, qemu-system-arm
wget
parted
mkfs.vfat, mkfs.ext4, mkswap
bsdtar
scp, ssh, ssh-keygen
pacman
The scripts also assume that you like vim :)
virtual machine start
---------------------
to open a shell into the created machine, run
$> ./start.sh [path to created image]
To boot a created virtual machine, run
$> sudo ./boot.sh [path to created image]
the start.sh script assumes that you want a throwaway session, so it will start
the virtual machine in snapshot mode and drop you into an ssh session. Once you
exit that session, the machine is shutdown and changes made to the image are
discarded. This behavior can be changed using the following environment
variables:
FOREGROUND : set this to 1 to start a qemu serial connection instead of a ssh
session. Useful to capture boot output.
PERSISTENT : set this to 1 to make persistent changes to the image that are
not discarded on shutdown.
The username and password for the created image is parabola:parabola, or
root:parabola respectively. If a packaging environment is setup, the system is
configured for passwordless sudo for the parabola user and the package tree and
a build chroot are prepared. have fun. also check out the .bashrc of the
parabola user in the created virtual machine, for batch build integration based
on task-spooler.
tarball creation
----------------
to create a tarball from the created vm image, run
$> sudo ./make_tarball.sh [path to created image]
the tarball creation script assumes to operate on an image *without* packaging
environment setup and will not perform additional cleanup operations if used on
the wrong image. Things that are cleaned up are:
/root/.ssh
/etc/ssh/ssh_host_*
/etc/pacman.d/gnupg
/var/log/*
/var/cache/*
/lost+found
The start.sh script assumes that you want a throwaway session, so it will start
the virtual machine in snapshot mode, and changes during the session will be
discarded.

134
boot.sh Executable file

@ -0,0 +1,134 @@
#!/bin/bash
##############################################################################
# parabola-imagebuilder #
# #
# Copyright (C) 2017 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
# this is a convenience script to start a created VM
##############################################################################
# common directories
startdir="$(pwd)"
export TOPBUILDDIR="$startdir"/build
export TOPSRCDIR="$startdir"/src
mkdir -p "$TOPBUILDDIR"
chown "$SUDO_USER" "$TOPBUILDDIR"
# shellcheck source=src/shared/common.sh
. "$TOPSRCDIR"/shared/common.sh
# sanity checks
if [ "$(id -u)" -ne 0 ]; then
die -e "$ERROR_INVOCATION" "must be root"
fi
# shellcheck source=src/qemu.sh
. "$TOPSRCDIR"/qemu.sh
check_kernel_arch() {
echo -n "checking for kernel name ..."
local kernel
kernel=$(find "$1" -maxdepth 1 -type f -iname '*vmlinu*' | head -n1)
[ -n "$kernel" ] || kernel=no
echo "$(basename "$kernel")"
[ "x$kernel" != "xno" ] || return
# check if the kernel has an elf header and extract arch
echo -n "checking for kernel elf header ... "
set -o pipefail
machine=$(readelf -h "$kernel" 2>/dev/null | grep Machine | awk '{print $2}') || machine=no
set +o pipefail
echo "$machine"
[ "x$machine" != "xno" ] && return
# no elf header? maybe arm?
echo -n "checking for ARM boot executable ... "
local is_arm=no
file "$kernel" | grep -q 'ARM boot executable' && is_arm=yes
echo "$is_arm"
[ "x$is_arm" == "xyes" ] && machine=ARM
[ "x$machine" != "xno" ] && return
# no idea, just bail.
error "unable to extract kernel arch from image"
return "$ERROR_MISSING"
}
qemu_setargs_arm() {
qemu_args=(
-snapshot
-nographic
-machine vexpress-a9
-cpu cortex-a9
-m 1G
-kernel "$1"/vmlinuz-linux-libre
-dtb "$1"/dtbs/linux-libre/vexpress-v2p-ca9.dtb
-initrd "$1"/initramfs-linux-libre.img
--append "console=ttyAMA0 rw root=/dev/mmcblk0p3"
-drive if=sd,driver=raw,cache=writeback,file="$2"
)
}
qemu_setargs_riscv64() {
qemu_args=(
-snapshot
-nographic
-machine virt
-m 2G
-kernel bbl
-append "console=ttyS0 rw root=/dev/vda"
-drive file="${3}p3",format=raw,id=hd0
-device virtio-blk-device,drive=hd0
-object rng-random,filename=/dev/urandom,id=rng0
-device virtio-rng-device,rng=rng0
-device virtio-net-device,netdev=usernet
-netdev user,id=usernet
)
}
boot_from_image() {
[ -f "$1" ] || die "$1: image does not exist"
local loopdev
qemu_img_losetup "$1" || return
# mount the boot partition
mkdir -p "$TOPBUILDDIR"/mnt
mount "${loopdev}p1" "$TOPBUILDDIR"/mnt || return
trap_add "umount -R $TOPBUILDDIR/mnt" INT TERM EXIT
local machine
check_kernel_arch "$TOPBUILDDIR"/mnt || return
case "$machine" in
RISC-V) arch=riscv64 ;;
ARM) arch=arm ;;
*) error "unrecognized machine '$machine'"
return "$ERROR_UNSPECIFIED" ;;
esac
qemu_args=()
"qemu_setargs_$arch" "$TOPBUILDDIR"/mnt "$1" "$loopdev"
QEMU_AUDIO_DRV=none "qemu-system-$arch" "${qemu_args[@]}"
}
boot_from_image "$1" || die "boot failed"

@ -1,8 +1,8 @@
#!/bin/bash
##############################################################################
# parabola-arm-imagebuilder #
# parabola-imagebuilder #
# #
# Copyright (C) 2017 Andreas Grapentin #
# Copyright (C) 2017, 2018 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
@ -18,48 +18,30 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
set -eu
die() { echo "$*" 1>&2 ; exit 1; }
# this script prepares an armv7h parabola image for use with start.sh
[ $(id -u) -ne 0 ] && die "must be root"
[ -z "${SUDO_USER:-}" ] && die "SUDO_USER not set"
export OUTFILE="${OUTFILE:-armv7h.img}"
# target options
export ARCH="${ARCH:-armv7h}"
export SIZE="${SIZE:-64G}"
export ARCHTARBALL="${ARCHTARBALL:-ArchLinuxARM-armv7-latest.tar.gz}"
export PARABOLATARBALL="${PARABOLATARBALL:-ParabolaARM-armv7-LATEST.tar.gz}"
export MIRROR="${MIRROR:-https://redirector.parabola.nu/\$repo/os/\$arch}"
export _builddir=build
mkdir -p "$_builddir"
chown $SUDO_USER "$_builddir"
# common directories
startdir="$(pwd)"
export TOPBUILDDIR="$startdir"/build
export TOPSRCDIR="$startdir"/src
mkdir -p "$TOPBUILDDIR"
chown "$SUDO_USER" "$TOPBUILDDIR"
export _outfile="$_builddir/$(basename "$OUTFILE")"
# shellcheck source=src/shared/common.sh
. "$TOPSRCDIR"/shared/common.sh
# prepare the empty image
./src/stage0.sh
if [ -n "${ARCHBOOTSTRAP:-}" ]; then
# install a clean archlinux-arm system in the empty image
wget -nc http://os.archlinuxarm.org/os/$ARCHTARBALL
TARBALL="$ARCHTARBALL" ./src/stage1.sh
# migrate the installed image to a clean parabola
./src/stage2.sh
else
# install a clean parabola-arm system in the empty image
wget -nc https://repo.parabola.nu/iso/arm/LATEST/$PARABOLATARBALL
TARBALL="$PARABOLATARBALL" ./src/stage1.sh
# sanity checks
if [ "$(id -u)" -ne 0 ]; then
die -e "$ERROR_INVOCATION" "must be root"
fi
# setup package development environment
[ -n "${DEVSETUP:-}" ] && ./src/stage3.sh
# shellcheck source=src/qemu.sh
. "$TOPSRCDIR"/qemu.sh
# cleanup
chown $SUDO_USER $_outfile
mv -v "$_outfile" "$OUTFILE"
rm -rf "$_builddir"
qemu_make_image "$TOPBUILDDIR/parabola-$ARCH.img" "$SIZE" \
|| die "failed to prepare qemu base image"
echo "all done :)"
msg "all done."

@ -1,66 +0,0 @@
#!/bin/bash
##############################################################################
# parabola-arm-imagebuilder #
# #
# Copyright (C) 2017 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
set -eu
die() { echo "$*" 1>&2 ; exit 1; }
[ $(id -u) -ne 0 ] && die "must be root"
_builddir=build
mkdir -p "$_builddir"
_imagefile="$_builddir/$(basename "$1")"
cp $1 $_imagefile
_rootdir="$_builddir"/root-$$
_loopdev=$(sudo losetup -f --show "$_imagefile")
sudo partprobe $_loopdev
# register a cleanup error handler
function cleanup {
sudo umount ${_loopdev}p1
sudo umount ${_loopdev}p3
sudo losetup -d $_loopdev
rm -rf "$_rootdir" "$_imagefile"
}
trap cleanup ERR
# mount the image
mkdir -p "$_rootdir"
sudo mount ${_loopdev}p3 "$_rootdir"
sudo mount ${_loopdev}p1 "$_rootdir"/boot
# clean the image
rm -fvr \
"$_rootdir"/root/.ssh \
"$_rootdir"/etc/ssh/ssh_host_* \
"$_rootdir"/var/log/* \
"$_rootdir"/var/cache/* \
"$_rootdir"/lost+found
# create the tarball
tar -czf ParabolaARM-armv7-$(date "+%Y-%m-%d").tar.gz -C "$_rootdir" .
# cleanup
sudo umount ${_loopdev}p1
sudo umount ${_loopdev}p3
sudo losetup -d $_loopdev
rm -rf "$_rootdir" "$_imagefile"

@ -1,52 +0,0 @@
#!/bin/bash
##############################################################################
# parabola-arm-imagebuilder #
# #
# Copyright (C) 2017 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
set -eu
set -x
die() { echo "$*" 1>&2 ; exit 1; }
_tarball=$1
# parse date from tarball
_date=$(echo "${_tarball%.tar.gz}" | rev | cut -d'-' -f1-3 | rev)
# create checksums
sha512sum $_tarball > SHA512SUMS
whirlpool-hash $_tarball > WHIRLPOOLSUMS
# sign tarball and checksum
gpg --detach-sign $_tarball
gpg --detach-sign SHA512SUMS
gpg --detach-sign WHIRLPOOLSUMS
# upload tarball and checksum
_repopath="/srv/repo/main/iso/arm/$_date"
ssh repo@repo "mkdir -p $_repopath"
scp $_tarball{,.sig} SHA512SUMS{,.sig} WHIRLPOOLSUMS{,.sig} repo@repo:$_repopath/
# update LATEST symlinks
ssh repo@repo "mkdir -p $_repopath/../LATEST"
for f in $_tarball{,.sig} SHA512SUMS{,.sig} WHIRLPOOLSUMS{,.sig}; do
ssh repo@repo "ln -fs ../$_date/$f $_repopath/../LATEST/$(echo $f | sed "s/$_date/LATEST/g")"
done
# cleanup
rm -rf $_tarball.sig SHA512SUMS{,.sig} WHIRLPOOLSUMS{,.sig}

150
src/qemu.sh Normal file

@ -0,0 +1,150 @@
#!/bin/bash
##############################################################################
# parabola-arm-imagebuilder #
# #
# Copyright (C) 2018 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
qemu_img_partition_and_mount_for_armv7h() {
parted -s "$1" \
mklabel gpt \
mkpart ESP fat32 1MiB 513MiB \
set 1 boot on \
mkpart primary linux-swap 513MiB 4609MiB \
mkpart primary ext4 4609MiB 100% || return
check_exe -r mkfs.vfat mkfs.ext4
mkfs.vfat -F 32 "${1}p1"
mkswap "${1}p2"
mkfs.ext4 "${1}p3"
mkdir -p "$2"
mount "${1}p3" "$2" || return
trap_add "umount -R $2" INT TERM EXIT
mkdir -p "$2"/boot
mount "${1}p1" "$2"/boot || return
}
qemu_img_partition_and_mount_for_riscv64() {
parted -s "$1" \
mklabel gpt \
mkpart primary ext2 1MiB 513MiB \
set 1 boot on \
mkpart primary linux-swap 513MiB 4609MiB \
mkpart primary ext4 4609MiB 100% || return
check_exe mkfs.ext2 mkfs.ext4
mkfs.ext2 "${1}p1"
mkswap "${1}p2"
mkfs.ext4 "${1}p3"
mkdir -p "$2"
mount "${1}p3" "$2" || return
trap_add "umount -R $2" INT TERM EXIT
mkdir -p "$2"/boot
mount "${1}p1" "$2"/boot || return
}
qemu_img_losetup() {
echo -n "checking for free loop device ... "
loopdev=$(losetup -f --show "$1") || loopdev=no
echo "$loopdev"
[ "x$loopdev" == "xno" ] && return "$ERROR_MISSING"
trap_add "qemu_img_lorelease $loopdev" INT TERM EXIT
}
qemu_img_lorelease() {
losetup -d "$1"
}
qemu_setup_user_static() {
# borrowed from /usr/bin/librechroot
local setarch interpreter
case "$ARCH" in
armv7h) setarch=armv7l; interpreter=/usr/bin/qemu-arm- ;;
*) setarch="$ARCH"; interpreter=/usr/bin/qemu-"$ARCH"- ;;
esac
if ! setarch "$setarch" /bin/true 2>/dev/null; then
# target arch can't execute natively, pacstrap is going to need help by qemu
# Make sure that qemu-static is set up with binfmt_misc
if [[ -z $(grep -l -F \
-e "interpreter $interpreter" \
-r -- /proc/sys/fs/binfmt_misc 2>/dev/null \
| xargs -r grep -xF 'enabled') ]]
then
error "unable to continue - need qemu-user-static for $ARCH"
return "$ERROR_MISSING"
fi
mkdir -p "$1"/usr/bin
cp -v "$interpreter"* "$1"/usr/bin || return
trap_add "qemu_cleanup_user_static $1"
fi
}
qemu_cleanup_user_static() {
rm -f "$1"/usr/bin/qemu-*
}
qemu_make_image() {
msg "preparing parabola qemu image for $ARCH"
# skip, if already exists
check_file "$1" && return
check_exe -r parted
# write to preliminary file
local tmpfile="$1.part"
rm -f "$tmpfile"
# create an empty image
qemu-img create -f raw "$tmpfile" "$2" || return
# create a minimal pacman.conf
cat > "$TOPBUILDDIR/pacman.conf.$ARCH" << EOF
[options]
Architecture = $ARCH
[libre]
Server = $MIRROR
[core]
Server = $MIRROR
[extra]
Server = $MIRROR
[community]
Server = $MIRROR
EOF
# setup the image (in a subshell for trap management)
(
loopdev=''
qemu_img_losetup "$tmpfile" || return
dd if=/dev/zero of="$loopdev" bs=1M count=8 || return
"qemu_img_partition_and_mount_for_$ARCH" "$loopdev" "$TOPBUILDDIR"/mnt || return
qemu_setup_user_static "$TOPBUILDDIR"/mnt || return
pacstrap -GMcd -C "$TOPBUILDDIR/pacman.conf.$ARCH" "$TOPBUILDDIR"/mnt || return
) || return
mv "$tmpfile" "$1"
}

100
src/shared/checks.sh Normal file

@ -0,0 +1,100 @@
#!/bin/bash
##############################################################################
# parabola-imagebuilder #
# #
# Copyright (C) 2018 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
check_exe() {
local OPTIND o r=
while getopts "r" o; do
case "$o" in
r) r=yes ;;
*) die -e "$ERROR_INVOCATION" "Usage: ${FUNCNAME[0]} [-r] program ..." ;;
esac
done
shift $((OPTIND-1))
local v res=0
for v in "$@"; do
echo -n "checking for $v in \$PATH ... "
local have_exe=yes
type -p "$v" >/dev/null || have_exe=no
echo $have_exe
if [ "x$have_exe" != "xyes" ]; then
[ "x$r" == "xyes" ] && die -e "$ERROR_MISSING" "missing $v in \$PATH"
res="$ERROR_MISSING"
fi
done
return "$res"
}
check_file() {
local OPTIND o r=
while getopts "r" o; do
case "$o" in
r) r=yes ;;
*) die -e "$ERROR_INVOCATION" "Usage: ${FUNCNAME[0]} [-r] file ..." ;;
esac
done
shift $((OPTIND-1))
local v res=0
for v in "$@"; do
echo -n "checking for $v ... "
local have_file=yes
[ -f "$v" ] || have_file=no
echo $have_file
if [ "x$have_file" != "xyes" ]; then
[ "x$r" == "xyes" ] && die -e "$ERROR_MISSING" "missing $v in filesystem"
res="$ERROR_MISSING"
fi
done
return "$res"
}
check_gpgkey() {
local OPTIND o r=
while getopts "r" o; do
case "$o" in
r) r=yes ;;
*) die -e "$ERROR_INVOCATION" "Usage: ${FUNCNAME[0]} [-r] key" ;;
esac
done
shift $((OPTIND-1))
local v res=0
for v in "$@"; do
echo -n "checking for key $v ... "
local have_key=yes
sudo -u "$SUDO_USER" gpg --list-keys "$v" &>/dev/null || have_key=no
echo $have_key
if [ "x$have_key" != "xyes" ]; then
[ "x$r" == "xyes" ] && die -e "$ERROR_MISSING" "missing $v in keyring"
res="$ERROR_MISSING"
fi
done
return "$res"
}

68
src/shared/common.sh Normal file

@ -0,0 +1,68 @@
#!/bin/bash
##############################################################################
# parabola-imagebuilder #
# #
# Copyright (C) 2018 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
# shellcheck source=src/shared/feedback.sh
. "$TOPSRCDIR"/shared/feedback.sh
# shellcheck source=src/shared/checks.sh
. "$TOPSRCDIR"/shared/checks.sh
retry() {
local OPTIND o n=5 s=60
while getopts "n:s:" o; do
case "$o" in
n) n="$OPTARG" ;;
s) s="$OPTARG" ;;
*) die -e $ERROR_INVOCATION "Usage: ${FUNCNAME[0]} [-n tries] [-s delay] cmd ..." ;;
esac
done
shift $((OPTIND-1))
for _ in $(seq "$((n - 1))"); do
"$@" && return 0
sleep "$s"
done
"$@" || return
}
# appends a command to a trap
# source: https://stackoverflow.com/questions/3338030/multiple-bash-traps-for-the-same-signal
#
# - 1st arg: code to add
# - remaining args: names of traps to modify
#
trap_add() {
trap_add_cmd=$1; shift || fatal "${FUNCNAME[0]} usage error"
for trap_add_name in "$@"; do
trap -- "$(
# helper fn to get existing trap command from output
# of trap -p
extract_trap_cmd() { printf '%s\n' "$3"; }
# print the new trap command
printf '%s\n' "${trap_add_cmd}"
# print existing trap command with newline
eval "extract_trap_cmd $(trap -p "${trap_add_name}")"
)" "${trap_add_name}" \
|| fatal "unable to add to trap ${trap_add_name}"
done
}
# set the trace attribute for the above function. this is
# required to modify DEBUG or RETURN traps because functions don't
# inherit them unless the trace attribute is set
declare -f -t trap_add

38
src/stage0.sh → src/shared/feedback.sh Executable file → Normal file

@ -1,8 +1,8 @@
#!/bin/bash
##############################################################################
# parabola-arm-imagebuilder #
# parabola-imagebuilder #
# #
# Copyright (C) 2017 Andreas Grapentin #
# Copyright (C) 2018 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
@ -18,8 +18,34 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
set -eu
# error codes
export ERROR_UNSPECIFIED=1
export ERROR_INVOCATION=2
export ERROR_MISSING=3
export ERROR_BUILDFAIL=4
export ERROR_KEYFAIL=5
# create an empty qemu image
rm -f $_outfile
qemu-img create -f raw $_outfile $SIZE
# messaging functions
msg() {
echo "$(tput bold)$(tput setf 2)==>$(tput setf 7) $*$(tput sgr0)";
}
error() {
echo "$(tput bold)$(tput setf 4)==> ERROR:$(tput setf 7) $*$(tput sgr0)" 1>&2
}
die() {
local OPTIND o e="$ERROR_UNSPECIFIED"
while getopts "e:" o; do
case "$o" in
e) e="$OPTARG" ;;
*) die -e "$ERROR_INVOCATION" "Usage: ${FUNCNAME[0]} [-e status] msg ..." ;;
esac
done
shift $((OPTIND-1))
error "$@"
trap - ERR
exit "$e"
}
trap 'die "unknown error"' ERR

@ -1,93 +0,0 @@
#!/bin/bash
##############################################################################
# parabola-arm-imagebuilder #
# #
# Copyright (C) 2017 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
set -eu
_bootdir="$_builddir/boot"
_rootdir="$_builddir/root"
_loopdev=$(losetup -f --show "$_outfile")
# setup an error exit handler for cleanup
function cleanup {
echo "exiting due to earlier errors..." >&2
for part in p1 p3; do
umount $_loopdev$part || true
done
losetup -d $_loopdev || true
rm -rf "$_bootdir" "$_rootdir"
rm -f "$_outfile"
}
trap cleanup ERR
# the following installation instructions are adapted from
# https://archlinuxarm.org/platforms/armv7/arm/versatile-express
# partition the image
dd if=/dev/zero of=$_loopdev bs=1M count=8
parted -s $_loopdev \
mklabel gpt \
mkpart ESP fat32 1MiB 513MiB \
set 1 boot on \
mkpart primary linux-swap 513MiB 4609MiB \
mkpart primary ext4 4609MiB 100%
# create filesystems
mkfs.vfat -F 32 ${_loopdev}p1
mkswap ${_loopdev}p2
mkfs.ext4 ${_loopdev}p3
# install the base image
mkdir -p "$_bootdir"
mkdir -p "$_rootdir"
mount ${_loopdev}p1 "$_bootdir"
mount ${_loopdev}p3 "$_rootdir"
bsdtar -vxpf $TARBALL -C "$_rootdir"
sync
# fill the boot partition and create fstab
mv -v "$_rootdir"/boot/* "$_bootdir"
cat >> "$_rootdir"/etc/fstab << EOF
/dev/mmcblk0p1 /boot vfat defaults 0 0
/dev/mmcblk0p2 none swap defaults 0 0
EOF
# create and install root ssh keys for access
mkdir -p keys
test -f keys/id_rsa || ssh-keygen -N '' -f keys/id_rsa
chown $SUDO_USER keys/id_rsa*
mkdir -m 700 "$_rootdir"/root/.ssh
install -m 600 -o 0 -g 0 keys/id_rsa.pub "$_rootdir"/root/.ssh/authorized_keys
# create and install ssh host keys
for cipher in dsa ecdsa ed25519 rsa; do
if [ ! -f keys/ssh_host_${cipher}_key ]; then
ssh-keygen -N '' -t ${cipher} -f keys/ssh_host_${cipher}_key
fi
install -m 600 -o 0 -g 0 keys/ssh_host_${cipher}_key "$_rootdir"/etc/ssh
install -m 644 -o 0 -g 0 keys/ssh_host_${cipher}_key.pub "$_rootdir"/etc/ssh
done
# tie up any loose ends
for part in p1 p3; do
umount $_loopdev$part
done
losetup -d $_loopdev
rm -rf "$_bootdir" "$_rootdir"

@ -1,133 +0,0 @@
#!/bin/bash
##############################################################################
# parabola-arm-imagebuilder #
# #
# Copyright (C) 2017 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
set -eu
_scriptfile="$_builddir"/migrate.sh
_pidfile="$_builddir"/qemu-$$.pid
_bootdir="$_builddir"/boot-$$
_loopdev=$(sudo losetup -f --show $_outfile)
# register cleanup handler to stop the started VM
function cleanup {
test -f "$_pidfile" && (kill -9 $(cat "$_pidfile") || true)
rm -f "$_pidfile"
umount ${_loopdev}p1
losetup -d $_loopdev
rm -rf "$_bootdir"
rm -f "$_scriptfile"
}
trap cleanup ERR
# create the migration script, adapted from
# https://wiki.parabola.nu/Migration_from_Arch_ARM
cat > "$_scriptfile" << 'EOF'
#!/bin/bash
set -eu
# install the keyrings and mirrorlist
sed -i 's/^SigLevel.*/SigLevel = Never/' /etc/pacman.conf
pacman --noconfirm -U https://www.parabola.nu/packages/libre/any/parabola-keyring/download/
pacman --noconfirm -U https://www.parabola.nu/packages/libre/any/archlinux32-keyring/download/
pacman --noconfirm -U https://www.parabola.nu/packages/core/any/archlinux-keyring/download/
pacman --noconfirm -U https://www.parabola.nu/packages/libre/any/pacman-mirrorlist/download/
pacman --noconfirm -S archlinuxarm-keyring
sed -i 's/^SigLevel.*/SigLevel = Required DatabaseOptional/' /etc/pacman.conf
# update the keyring
pacman-key --init
pacman-key --populate archlinuxarm archlinux archlinux32 parabola
pacman-key --refresh-keys
# install the mirrorlist
[ -f /etc/pacman.d/mirrorlist.pacnew ] && mv /etc/pacman.d/mirrorlist{.pacnew,}
# enable the [libre] and disable [alarm] in pacman.conf
sed -i '/^\[core\]/i \
[libre] \
Include = /etc/pacman.d/mirrorlist \
' /etc/pacman.conf
sed -Ei '/^\[alarm\]|\[aur\]/,+2d' /etc/pacman.conf
# clear the pacman cache. all of it.
yes | pacman -Scc
# fix the architecture in /etc/pacman.conf
sed -i 's/^Architecture.*/Architecture = armv7h/' /etc/pacman.conf
# update the system to parabola
pacman --noconfirm -Syy
pacman --noconfirm -S pacman
mv /etc/pacman.conf{.pacnew,}
pacman --noconfirm -Syyuu
pacman --noconfirm -S your-freedom
yes | pacman -S linux-libre
# cleanup users
userdel -r alarm
useradd -mU parabola
echo 'parabola:parabola' | chpasswd
echo 'root:parabola' | chpasswd
# cleanup hostname
echo "parabola-arm" > /etc/hostname
# enable UTF-8 locale
sed -i 's/#en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen
locale-gen
sed -i 's/LANG.*/LANG=en_US.UTF-8/' /etc/locale.conf
EOF
chmod +x "$_scriptfile"
# start the VM
mkdir -p "$_bootdir"
mount ${_loopdev}p1 $_bootdir
QEMU_AUDIO_DRV=none qemu-system-arm \
-M vexpress-a9 \
-m 1G \
-dtb "$_bootdir"/dtbs/vexpress-v2p-ca9.dtb \
-kernel "$_bootdir"/zImage \
--append "root=/dev/mmcblk0p3 rw roottype=ext4 console=ttyAMA0" \
-drive if=sd,driver=raw,cache=writeback,file="$_outfile" \
-display none \
-net user,hostfwd=tcp::2022-:22 \
-net nic \
-daemonize \
-pidfile "$_pidfile"
# wait for ssh to be up
while ! ssh -p 2022 -i keys/id_rsa root@localhost -o StrictHostKeyChecking=no true 2>/dev/null; do
echo -n . && sleep 5
done && echo
# copy and execute the migration script
scp -P 2022 -i keys/id_rsa "$_scriptfile" root@localhost:
ssh -p 2022 -i keys/id_rsa root@localhost "./$(basename "$_scriptfile")"
# stop the VM
ssh -p 2022 -i keys/id_rsa root@localhost "nohup shutdown -h now &>/dev/null & exit"
while kill -0 $(cat "$_pidfile") 2> /dev/null; do echo -n . && sleep 5; done && echo
# cleanup
umount ${_loopdev}p1
losetup -d $_loopdev
rm -rf "$_bootdir" "$_scriptfile" "$_pidfile"

@ -1,167 +0,0 @@
#!/bin/bash
##############################################################################
# parabola-arm-imagebuilder #
# #
# Copyright (C) 2017 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
set -eu
_scriptfile="$_builddir"/migrate.sh
_pidfile="$_builddir"/qemu-$$.pid
_bootdir="$_builddir"/boot-$$
_loopdev=$(sudo losetup -f --show "$_outfile")
# register cleanup handler to stop the started VM
function cleanup {
test -f "$_pidfile" && (kill -9 $(cat "$_pidfile") || true)
rm -f "$_pidfile"
umount ${_loopdev}p1
losetup -d $_loopdev
rm -rf "$_bootdir"
rm -f "$_scriptfile"
}
trap cleanup ERR
# create the package build preparation script, adapted from
# https://wiki.parabola.nu/Package_maintainer_guide
(source /etc/makepkg.conf && cat > "$_scriptfile" << EOF
#!/bin/bash
set -eu
# setup parabola login keys
cat /root/.ssh/authorized_keys >> /home/parabola/.ssh/authorized_keys
# fix key permissions and ownership
chown -R parabola:parabola /home/parabola/{.gnupg,.ssh,.gitconfig}
chmod 600 /home/parabola/.ssh/authorized_keys
# install needed packages
pacman --noconfirm -S libretools base-devel vim sudo \
rxvt-unicode-terminfo bash-completion htop
# update configuration
sed -i \
-e 's_^#PKGDEST.*_PKGDEST="/home/parabola/output/packages"_' \
-e 's_^#SRCDEST.*_SRCDEST="/home/parabola/output/sources"_' \
-e 's_^#SRCPKGDEST.*_SRCPKGDEST="/home/parabola/output/srcpackages"_' \
-e 's_^#LOGDEST.*_LOGDEST="/home/parabola/output/makepkglogs"_' \
-e 's_^#PACKAGER.*_PACKAGER="$PACKAGER"_' \
-e 's_^#GPGKEY.*_GPGKEY="$GPGKEY"_' \
/etc/makepkg.conf
sed -i \
-e 's_^CHROOTDIR.*_CHROOTDIR="/home/parabola/build"_' \
-e 's_^CHROOTEXTRAPKG.*_CHROOTEXTRAPKG=(vim)_' \
/etc/libretools.d/chroot.conf
sed -i \
-e 's_^HOOKPOSTRELEASE.*_HOOKPOSTRELEASE=""_' \
/etc/libretools.conf
# create directories
mkdir -p /home/parabola/output/{packages,sources,srcpackages,makepkglogs}
chown -R parabola:parabola /home/parabola/output
# disable systemd-stdin hack...
sed -i '/XXX: SYSTEMD-STDIN HACK/,+9d' /usr/bin/librechroot
# setup sudo
cat > /etc/sudoers.d/parabola << IEOF
# grant full permissions to user parabola
parabola ALL=(ALL) NOPASSWD: ALL
IEOF
# setup work directories
su - parabola -c createworkdir
su - parabola -c "sudo librechroot make"
# setup batch building
pacman --noconfirm -S task-spooler
cat >> /home/parabola/.bashrc << 'IEOF'
alias sudo='sudo '
function librespool() {
local cmd
printf -v cmd '%q ' "\$@"
tsp -d script --return --quiet --command "\$cmd" /dev/null
}
alias librechroot-spool='librespool sudo /usr/bin/librechroot'
alias libremakepkg-spool='librespool sudo /usr/bin/libremakepkg'
alias qbuild='if tsp | grep " \$(pwd)\\$" >/dev/null; then tspr; fi && tsp echo \$(pwd) && librechroot-spool update && libremakepkg-spool && tsp -d librestage'
alias tspr='d=\$(tsp | grep " \$(pwd)\\$" | head -n1 | cut -d" " -f1) && for i in \$(seq \$d \$((\$d+3))); do tsp -r \$i; done'
alias tspl='watch -n5 tsp'
alias tspc='while tsp | grep -q running; do tsp -c; done'
alias librecommit='if tsp | grep " \$(pwd)\\$" >/dev/null; then tspr; fi && git commit -m "\$(pwd | rev | cut -d"/" -f1-2 | rev): updated to \$(bash -c "source PKGBUILD && echo \\\$pkgver")"'
IEOF
EOF
)
chmod +x "$_scriptfile"
# start the VM
mkdir -p "$_bootdir"
mount ${_loopdev}p1 "$_bootdir"
_board="vexpress-a9"
_cpu="cortex-a9"
_memory="1G"
_kernel="$_bootdir"/vmlinuz-linux-libre
_dtb="$_bootdir"/dtbs/linux-libre/vexpress-v2p-ca9.dtb
_initrd="$_bootdir"/initramfs-linux-libre.img
QEMU_AUDIO_DRV=none qemu-system-arm \
-M $_board \
-cpu $_cpu \
-m $_memory \
-kernel "$_kernel" \
-dtb "$_dtb" \
-initrd "$_initrd" \
--append "root=/dev/mmcblk0p3 rw roottype=ext4 console=ttyAMA0" \
-drive if=sd,driver=raw,cache=writeback,file="$_outfile" \
-display none \
-net user,hostfwd=tcp::2022-:22 \
-net nic \
-daemonize \
-pidfile "$_pidfile"
# wait for ssh to be up
while ! ssh -p 2022 -i keys/id_rsa root@localhost -o StrictHostKeyChecking=no true 2>/dev/null; do
echo -n . && sleep 5
done && echo
# copy the current users keys to the VM
scp -rP 2022 -i keys/id_rsa "$(sudo -iu $SUDO_USER pwd)"/.gnupg root@localhost:/home/parabola/
scp -rP 2022 -i keys/id_rsa "$(sudo -iu $SUDO_USER pwd)"/.ssh root@localhost:/home/parabola/
scp -rP 2022 -i keys/id_rsa "$(sudo -iu $SUDO_USER pwd)"/.gitconfig root@localhost:/home/parabola/
# copy and execute the migration script
scp -P 2022 -i keys/id_rsa "$_scriptfile" root@localhost:
ssh -p 2022 -i keys/id_rsa root@localhost "./$(basename "$_scriptfile")"
# stop the VM
ssh -p 2022 -i keys/id_rsa root@localhost "nohup shutdown -h now &>/dev/null & exit"
while kill -0 $(cat "$_pidfile") 2> /dev/null; do echo -n . && sleep 5; done && echo
# cleanup
umount ${_loopdev}p1
losetup -d $_loopdev
rm -rf "$_bootdir" "$_scriptfile" "$_pidfile"

@ -1,94 +0,0 @@
#!/bin/bash
##############################################################################
# parabola-arm-imagebuilder #
# #
# Copyright (C) 2017 Andreas Grapentin #
# #
# This program is free software: you can redistribute it and/or modify #
# it under the terms of the GNU General Public License as published by #
# the Free Software Foundation, either version 3 of the License, or #
# (at your option) any later version. #
# #
# This program is distributed in the hope that it will be useful, #
# but WITHOUT ANY WARRANTY; without even the implied warranty of #
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the #
# GNU General Public License for more details. #
# #
# You should have received a copy of the GNU General Public License #
# along with this program. If not, see <http://www.gnu.org/licenses/>. #
##############################################################################
set -eu
_builddir=build
mkdir -p "$_builddir"
_imagefile=$1
_pidfile="$_builddir"/qemu-$$.pid
_bootdir="$_builddir"/boot-$$
_loopdev=$(sudo losetup -f --show "$_imagefile")
sudo partprobe $_loopdev
touch "$_pidfile"
# register a cleanup error handler
function cleanup {
test -f "$_pidfile" && (sudo kill -9 $(cat "$_pidfile") || true)
rm -f "$_pidfile"
sudo umount ${_loopdev}p1
sudo losetup -d $_loopdev
rm -rf "$_bootdir"
}
trap cleanup ERR
# start the VM
mkdir -p "$_bootdir"
sudo mount ${_loopdev}p1 "$_bootdir"
_board="vexpress-a9"
# FIXME: archlinuxarm rust SIGILLs on cortex-a9 cpus, using cortex-a15 for now
_cpu="cortex-a15"
_memory="1G"
_snapshot=""
[ -z "${PERSISTENT:-}" ] && _snapshot="-snapshot"
_daemonize="-nographic -serial mon:stdio"
[ -z "${FOREGROUND:-}" ] && _daemonize="-daemonize -pidfile $_pidfile -net user,hostfwd=tcp::2022-:22 -net nic -display none"
if [ -f "$_bootdir"/zImage ]; then
_kernel="$_bootdir"/zImage
_dtb="$_bootdir"/dtbs/vexpress-v2p-ca9.dtb
_initrd="$_bootdir"/initramfs-linux.img
else
_kernel="$_bootdir"/vmlinuz-linux-libre
_dtb="$_bootdir"/dtbs/linux-libre/vexpress-v2p-ca9.dtb
_initrd="$_bootdir"/initramfs-linux-libre.img
fi
QEMU_AUDIO_DRV=none qemu-system-arm \
-M $_board \
-cpu $_cpu \
-m $_memory \
-kernel "$_kernel" \
-dtb "$_dtb" \
-initrd "$_initrd" \
--append "root=/dev/mmcblk0p3 rw roottype=ext4 console=ttyAMA0" \
-drive if=sd,driver=raw,cache=writeback,file="$_imagefile" \
$_daemonize \
$_snapshot
if [ -z "${FOREGROUND:-}" ]; then
# wait for ssh to be up
_sshopts="-o StrictHostKeyChecking=no -o ConnectTimeout=5"
while ! ssh -p 2022 -i keys/id_rsa root@localhost $_sshopts true 2>/dev/null; do
echo -n . && sleep 5
done && echo
# open a session
ssh -p 2022 -i keys/id_rsa parabola@localhost
# shutdown the VM
ssh -p 2022 -i keys/id_rsa root@localhost "nohup shutdown -h now &>/dev/null & exit"
while sudo kill -0 $(cat "$_pidfile") 2> /dev/null; do echo -n . && sleep 5; done && echo
fi
# cleanup
sudo umount ${_loopdev}p1
sudo losetup -d $_loopdev
rm -rf "$_bootdir" "$_pidfile"