mirror of https://github.com/lineageos4microg/docker-lineage-cicd synced 2024-11-09 10:09:56 +01:00

Docker microservice for LineageOS Continuous Integration and Continous Deployment

Go to file

Pete Fotheringham f52d011a5e Correct path to outfile		2023-09-17 20:26:20 +01:00
.github	Bump docker/build-push-action from 3.3.0 to 4.0.0	2023-02-03 19:48:30 +01:00
src	Correct path to outfile	2023-09-17 20:26:20 +01:00
.gitignore	gitignore vscode config	2022-03-16 20:38:17 +01:00
apt_preferences	Add multibranch support	2017-12-13 12:13:36 +01:00
Dockerfile	add repo init args	2023-06-25 01:56:35 +00:00
LICENSE	Change license to GPLv3	2017-06-06 09:12:26 +02:00
README.md	Docs - document the necessary call to `docker pull`	2023-09-17 16:48:42 +01:00

README.md

docker-lineage-cicd

Docker microservice for LineageOS Continuous Integration and Continuous Deployment

Why Docker?

A fair number of dependencies is needed to build LineageOS, plus a Linux system (and a discrete knowledge of it). With Docker we give you a minimal Linux build system with all the tools and scripts already integrated, easing considerably the creation of your own LineageOS build.

Moreover Docker runs also on Microsoft Windows and Mac OS, which means that LineageOS can be built on such platforms without requiring a dual boot system or a manual set up of a Virtual Machine.

How do I install Docker?

The official Docker guides are well-written:

Linux (Ubuntu, Debian, CentOS and Fedora are officially supported)
Windows 10/Windows Server 2016 64bit
Mac OS El Capitan 10.11 or newer

If your Windows or Mac system doesn't satisfy the requirements (or if you have Oracle VirtualBox installed, you can use Docker Toolbox. Docker Toolbox is not described in this guide, but it should be very similar to the standard Docker installation.

Once you can run the hello-world image you're ready to start!

What does Docker build

Docker will produce two files in the zips directory:

The main ROM zip file e.g. lineage-20.0-20230702-microG-<device-name>.zip. This file can be flashed from recovery as described in the next section.
A -img.zip file e.g. lineage-20.0-20230702-microG-<device-name>-img.zip, that can usually be flashed using up-to-date versions of fastboot.

(Note that the process of flashing using fastboot is not documented, is untested, and is done at your own risk.)

How do I install the LineageOS for MicroG ROM

Follow the LineageOS installation instructions for your device, which can be accessed from the LineageOS Devices wiki pages. If the LineageOS installation instructions require or refer to any .img files, these images can be obtained by unzipping the -img.zip file mentioned in the previous section.

'Clean' and 'dirty' flashing

A 'clean' flash is when the data partition is wiped and/or formatted before the ROM is installed. This will remove all user-installed apps and data. It is sometimes referred to as a 'fresh installation'.

A 'dirty flash' is when the data partition is not wiped and/or formatted before the ROM is installed. Normally this will result in all user-installed apps and data still being present after the intallation.

Newer versions of the LineageOS for MicroG ROM can usually be 'dirty flashed' over older versions with the same Android version.

Dirty flashing is sometimes possible over

older versions of the LineageOS for MicroG ROM with an earlier Android version**;
the official LineageOS ROM (without microG)

In both these cases, problems may be encountered with app permissions, both for user-installed apps and for the pre-installed apps. These problems can sometimes be fixed by manually changing the app permissions.

If you are 'dirty' flashing, it is a good idea to backup your user-installed apps and data in case the 'dirty' flash fails.

How can I build LineageOS?

Before you start, make sure you have the latest version of our Docker image:

docker pull lineageos4microg/docker-lineage-cicd

This Docker image contains a great number of settings, to allow you to fully customize your LineageOS build. Here you can find all of them, with the default values between the brackets.

TL;DR - go to the Examples

Fundamental settings

The two fundamental settings are:

BRANCH_NAME (lineage-16.0): LineageOS branch, see the branch list here (multiple comma-separated branches can be specified)
DEVICE_LIST: comma-separated list of devices to build

Running a build with only these two set will create a ZIP file almost identical to the LineageOS official builds, just signed with the test keys.

When multiple branches are selected, use DEVICE_LIST_<BRANCH_NAME> to specify the list of devices for each specific branch (see the examples).

GMS / microG

To include microG (or possibly the actual Google Mobile Services) in your build, LineageOS expects certain Makefiles in vendor/partner_gms and variable WITH_GMS set to true.

This repo contains the common packages included for official lineageos4microg builds. To include it in your build, create an XML (the name is irrelevant, as long as it ends with .xml) in the /home/user/manifests folder with this content:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
    <project path="vendor/partner_gms" name="lineageos4microg/android_vendor_partner_gms" remote="github" revision="master" />
</manifest>

Additional custom apps

If you wish to add other apps to your ROM, you can include a repository with source code or prebuilt APKs. For prebuilt apks, see the android_vendor_partner_gms repository for examples on how the Android.mk file should look like.

Include the repo with another manifest file like this:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <project name="your-github-user/your-repo" path="prebuilts/my-custom-apps" remote="github" revision="master" />
</manifest>

And when starting the build, set the CUSTOM_PACKAGES variable to a list of app names (defined by LOCAL_MODULE in Android.mk) separated by spaces.

Signature spoofing

There are two options for the signature spoofing patch required for microG:

"Original" patches
Restricted patches

With the "original" patch the FAKE_SIGNATURE permission can be granted to any user app: while it may seem handy, this is considered dangerous by a great number of people, as the user could accidentally give this permission to rogue apps.

A more strict option is the restricted patch, where the FAKE_SIGNATURE permission can be obtained only by privileged system apps, embedded in the ROM during the build process.

The signature spoofing patch can be optionally included with:

SIGNATURE_SPOOFING (no): yes to use the original patch, restricted for the restricted one, no for none of them

If in doubt, use restricted: note that packages that requires the FAKE_SIGNATURE permission must be included in the build as system apps (e.g. as part of GMS or CUSTOM_PACKAGES)

Proprietary files

Some proprietary files are needed to create a LineageOS build, but they're not included in the LineageOS repo for legal reasons. You can obtain these blobs in three ways:

by pulling them from a running LineageOS
by extracting them from a LineageOS ZIP
by downloading them from TheMuppets GitHub and GitLab repositories (unofficial)

The third way is the easiest one and is enabled by default; if you're OK with that just move on, otherwise set INCLUDE_PROPRIETARY (true) to false and manually provide the blobs (not explained in this guide).

Over the Air updates

To enable OTA for you builds, you need to run a server that speaks the protocol understood by the LineageOS updater app and provide the URL to this server as OTA_URL variable for the build.

One implementation is LineageOTA, which is also available as Docker image. Follow these steps to prepare your builds for OTA:

Run the Docker image julianxhokaxhiu/lineageota
- Port 80 exposed to the internet (might want to add an HTTPS reverse proxy)
- The /srv/zips directory/volume of the CICD image mounted at /var/www/html/builds/full (can be read-only)
Set environment variables when building
- ZIP_SUBDIR to false
- OTA_URL to the address of the OTA server, with /api appended

If you don't setup a OTA server you won't be able to update the device from the updater app (but you can still update it manually with the recovery of course).

Signing

By default, builds are signed with the Android test keys. If you want to sign your builds with your own keys (highly recommended):

SIGN_BUILDS (false): set to true to sign the builds with the keys contained in /srv/keys; if no keys are present, a new set will be generated

Other settings

Other useful settings are:

CCACHE_SIZE (50G): change this if you want to give more (or less) space to ccache
WITH_SU (false): set to true to embed su in the build (note that, even when set to false, you can still enable root by flashing the su installable ZIP). This is only for lineage version 16 and below.
RELEASE_TYPE (UNOFFICIAL): change the release type of your builds
BUILD_TYPE (userdebug): type of your builds, see Android docs
BUILD_OVERLAY (false): normally each build is done on the source tree, then the tree is cleaned with mka clean. If you want to be sure that each build is isolated from the others, set BUILD_OVERLAY to true (longer build time). Requires --cap-add=SYS_ADMIN.
LOCAL_MIRROR (false): change this to true if you want to create a local mirror of the LineageOS source (> 200 GB)
CRONTAB_TIME (now): instead of building immediately and exit, build at the specified time (uses standard cron format)
ZIP_SUBDIR (true): Move the resulting zips to $ZIP_DIR/$codename instead of $ZIP_DIR/
PARALLEL_JOBS: Limit the number of parallel jobs to run (-j for repo sync and mka). By default, the build system should match the number of parallel jobs to the number of cpu cores on your machine. Reducing this number can help keeping it responsive for other tasks.

The full list of settings, including the less interesting ones not mentioned in this guide, can be found in the Dockerfile.

Volumes

You also have to provide Docker some volumes, where it'll store the source, the resulting builds, the cache and so on. The volumes are:

/srv/src, for the LineageOS sources
/srv/zips, for the output builds
/srv/logs, for the output logs
/srv/ccache, for the ccache
/srv/local_manifests, for custom manifests (optional)
/srv/userscripts, for the user scripts (optional)

When SIGN_BUILDS is true

/srv/keys, for the signing keys

When BUILD_OVERLAY is true

/srv/tmp, for temporary files

When LOCAL_MIRROR is true:

/srv/mirror, for the LineageOS mirror

Examples

Build for river (lineage-18.1, officially supported), test keys, no patches

docker run \
    -e "BRANCH_NAME=lineage-18.1" \
    -e "DEVICE_LIST=river" \
    -v "/home/user/lineage:/srv/src" \
    -v "/home/user/zips:/srv/zips" \
    -v "/home/user/logs:/srv/logs" \
    -v "/home/user/cache:/srv/ccache" \
    lineageos4microg/docker-lineage-cicd

Build for bacon (lineage-17.1, officially supported), custom keys, restricted signature spoofing with integrated microG and FDroid

docker run \
    -e "BRANCH_NAME=lineage-17.1" \
    -e "DEVICE_LIST=bacon" \
    -e "SIGN_BUILDS=true" \
    -e "SIGNATURE_SPOOFING=restricted" \
    -e "WITH_GMS=true" \
    -v "/home/user/lineage:/srv/src" \
    -v "/home/user/zips:/srv/zips" \
    -v "/home/user/logs:/srv/logs" \
    -v "/home/user/cache:/srv/ccache" \
    -v "/home/user/keys:/srv/keys" \
    -v "/home/user/manifests:/srv/local_manifests" \
    lineageos4microg/docker-lineage-cicd

If there are already keys in /home/user/keys they will be used, otherwise a new set will be generated before starting the build (and will be used for every subsequent build).

The microG and FDroid packages are not present in the LineageOS repositories, and must be provided e.g. through android_vendor_partner_gms.

Build for four devices on lineage-17.1 and lineage-18.1 (officially supported), custom keys, restricted signature spoofing with integrated microG and FDroid, custom OTA server

docker run \
    -e "BRANCH_NAME=lineage-17.1,lineage-18.1" \
    -e "DEVICE_LIST_LINEAGE_17_1=bacon,oneplus2" \
    -e "DEVICE_LIST_LINEAGE_18_1=river,lake" \
    -e "SIGN_BUILDS=true" \
    -e "SIGNATURE_SPOOFING=restricted" \
    -e "WITH_GMS=true" \
    -e "OTA_URL=https://api.myserver.com/" \
    -v "/home/user/lineage:/srv/src" \
    -v "/home/user/zips:/srv/zips" \
    -v "/home/user/logs:/srv/logs" \
    -v "/home/user/cache:/srv/ccache" \
    -v "/home/user/keys:/srv/keys" \
    -v "/home/user/manifests:/srv/local_manifests" \
    lineageos4microg/docker-lineage-cicd

Build for a6000 (not officially supported), custom keys, restricted signature spoofing with integrated microG and FDroid

As there is no official support for this device, we first have to include the sources in the source tree through an XML in the /home/user/manifests folder; from this thread we get the links of:

Device tree: https://github.com/dev-harsh1998/android_device_lenovo_a6000
Common Tree: https://github.com/dev-harsh1998/android_device_lenovo_msm8916-common
Kernel: https://github.com/dev-harsh1998/kernel_lenovo_msm8916
Vendor blobs: https://github.com/dev-harsh1998/proprietary-vendor_lenovo

Then, with the help of lineage.dependencies from the device tree and the common tree we create an XML /home/user/manifests/a6000.xml with this content:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <project name="dev-harsh1998/android_device_lenovo_a6000" path="device/lenovo/a6000" remote="github" />
  <project name="dev-harsh1998/android_device_lenovo_msm8916-common" path="device/lenovo/msm8916-common" remote="github" />
  <project name="dev-harsh1998/kernel_lenovo_msm8916" path="kernel/lenovo/a6000" remote="github" />
  <project name="dev-harsh1998/proprietary-vendor_lenovo" path="vendor/lenovo" remote="github" />
  <project name="LineageOS/android_device_qcom_common" path="device/qcom/common" remote="github" />
</manifest>

We also want to include microG so, like before, create an XML (for example /home/user/manifests/microg.xml) with this content:

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
    <project path="vendor/partner_gms" name="lineageos4microg/android_vendor_partner_gms" remote="github" revision="master" />
</manifest>

We also set INCLUDE_PROPRIETARY=false, as the proprietary blobs are already provided by the repo https://github.com/dev-harsh1998/prorietary_vendor_lenovo (so we don't have to include the TheMuppets repo).

Now we can just run the build like it was officially supported:

docker run \
    -e "BRANCH_NAME=lineage-15.1" \
    -e "DEVICE_LIST=a6000" \
    -e "SIGN_BUILDS=true" \
    -e "SIGNATURE_SPOOFING=restricted" \
    -e "WITH_GMS=true" \
    -e "INCLUDE_PROPRIETARY=false" \
    -v "/home/user/lineage:/srv/src" \
    -v "/home/user/zips:/srv/zips" \
    -v "/home/user/logs:/srv/logs" \
    -v "/home/user/cache:/srv/ccache" \
    -v "/home/user/keys:/srv/keys" \
    -v "/home/user/manifests:/srv/local_manifests" \
    lineageos4microg/docker-lineage-cicd