mirror/docker-lineage-cicd
1
0
Fork 0
mirror of https://github.com/lineageos4microg/docker-lineage-cicd synced 2024-11-09 10:09:56 +01:00
Code Issues

Rewrite README.md

Browse Source
This commit is contained in:
Nicola Corna 2018-03-08 08:00:01 +01:00
parent 5ffe75049a
commit 1a467b6ddd
1 changed files with 239 additions and 58 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

297
README.md
View File

@ -2,88 +2,269 @@
Docker microservice for LineageOS Continuous Integration and Continous Deployment
This branch contains a version suitable to build for multiple devices on different branches with the same Docker image: to do so it is necessary to download the full LineageOS mirror, about 200 GB. You probably want the more simple version in the master branch.
## Why Docker?
## Requirements
A fair number of dependencies is needed to build LineageOS, plus a Linux system
(and a discrete knowledge of it). With Docker we give you a minimal Linux build
system with all the tools and scripts already integrated, easing considerably
the creation of your own LineageOS build.
- At least Dual Core CPU (Higher is better)
- At least 6GB RAM (Higher is better)
- At least 500GB HDD Space (Higher is better)
Moreover Docker runs also on Microsoft Windows and Mac OS, which means that
LineageOS can be built on such platforms without requiring a dual boot system
or a manual set up of a Virtual Machine.
### Android propretary binaries
## How do I install Docker?
By default when you build Android from scratch you need to pull the binaries of your interested device via ADB. When the INCLUDE_PROPRIETARY variable is set to true (default), the proprietary binaries are downloaded automatically from [TheMuppets repository](https://github.com/TheMuppets); if you want to provide your own binaries, set this variable to false and add a corresponding XML in the local_manifests volume.
The official Docker guides are well-written:
* Linux ([Ubuntu](https://docs.docker.com/install/linux/docker-ce/ubuntu/), [Debian](https://docs.docker.com/install/linux/docker-ce/debian/), [CentOS](https://docs.docker.com/install/linux/docker-ce/centos/) and [Fedora](https://docs.docker.com/install/linux/docker-ce/fedora/) are officially supported)
* [Windows 10/Windows Server 2016 64bit](https://docs.docker.com/docker-for-windows/install/)
* [Mac OS El Capitan 10.11 or newer](https://docs.docker.com/docker-for-mac/install/)
## How it works
If your Windows or Mac system doesn't satisfy the requirements (or if you have
Oracle VirtualBox installed, you can use [Docker Toolbox](https://docs.docker.com/toolbox/overview/).
Docker Toolbox is not described in this guide, but it should be very similar to
the standard Docker installation.
This docker will autobuild any device list given for a specified branch every midnight at 02:00 UTC. In the end, any built ZIP will be moved to the relative volume mapped directory to `/srv/zips`.
Once you can run the [`hello-world` image](https://docs.docker.com/get-started/#test-docker-installation) you're ready to start!
> **IMPORTANT:** Remember to use VOLUME mapping. By default Docker creates container with max 10GB of Space. If you will not map volumes, the docker will just break during Source syncronization!
## How can I build LineageOS?
**NOTE:** `/home/user/local_manifests/` may contain multiple XMLs, since all the files will be then copied inside `.repo/local_manifests/`
This Docker image contains a great number of settings, to allow you to fully
customize your LineageOS build.
## Configuration
TL;DR - go to the [Examples](#examples)
You can configure the Docker by passing custom environment variables to it. See the [Dockerfile](Dockerfile#L11) for more details.
### Fundamental settings
## How to use
The two fundamental settings are (default value in brackets):
Specify the branches and the devices in the corresponding variables, separated by a comma.
* `BRANCH_NAME (cm-14.1)`: LineageOS branch, see the branch list [here](https://github.com/LineageOS/android_vendor_cm/branches)
* `DEVICE_LIST`: comma-separated list of devices to build
Running a build with only these two set will create a ZIP file almost identical
to the LineageOS official builds, just signed with the test keys.
### Signature spoofing
There are two options for the [signature spoofing patch](https://github.com/microg/android_packages_apps_GmsCore/wiki/Signature-Spoofing)
required for [microG](https://microg.org/):
* "Original" [patches](https://github.com/lineageos4microg/docker-lineage-cicd/tree/master/src/signature_spoofing_patches)
* Restricted patches
With the "original" patch the FAKE_SIGNATURE permission can be granted to any
user app: while it may seem handy, this is considered dangerous by a great
number of people, as the user could accidentally give this permission to rogue
apps.
A more strict option is the restricted patch, where the FAKE_SIGNATURE
permission can be obtained only by privileged system apps, embedded in the ROM
during the build process.
The signature spoofing patch can be optionally included with:
* `SIGNATURE_SPOOFING (no)`: `yes` to use the original patch, `restricted` for the restricted one, `no` for none of them
If in doubt, use `restricted`: note that packages that requires the
FAKE_SIGNATURE permission must be embedded in the build by adding them in
* `CUSTOM_PACKAGES`
Extra packages can be included in the tree by adding the corresponding manifest
XML to the local_manifests volume.
### Proprietary files
Some proprietary files are needed to create a LineageOS build, but they're not
included in the LineageOS repo for legal reasons. You can obtain these blobs in
three ways:
* by [pulling them from a running LineageOS](https://wiki.lineageos.org/devices/bacon/build#extract-proprietary-blobs)
* by [extracting them from a LineageOS ZIP](https://wiki.lineageos.org/extracting_blobs_from_zips.html)
* by downloading them from [TheMuppets repos](https://github.com/TheMuppets/manifests) (unofficial)
The third way is the easiest one and is enabled by default; if you're OK with
that just move on, otherwise set `INCLUDE_PROPRIETARY (true)` to `false` and
manually provide the blobs (not explained in this guide).
### OTA
If you have a server and you want to enable [OTA updates](https://github.com/julianxhokaxhiu/LineageOTA)
you have to provide the URL of your server during the build process with:
* `OTA_URL`
If you don't setup a OTA server you won't be able to update the device from the
updater app (but you can still update it manually with the recovery of course).
### Signing
By default, builds are signed with the Android test keys. If you want to sign
your builds with your own keys (highly recommended):
* `SIGN_BUILDS (false)`: set to `true` to sign the builds with the keys contained in `/srv/keys`; if no keys are present, a new set will be generated
### Other settings
Other useful settings are:
* `CCACHE_SIZE (50G)`: change this if you want to give more (or less) space to ccache
* `WITH_SU (false)`: set to `true` to embed `su` in the build (note that, even when set to `false`, you can still enable root by flashing the [su installable ZIP](https://download.lineageos.org/extras))
* `RELEASE_TYPE (UNOFFICIAL)`: change the release type of your builds
* `BUILD_OVERLAY (false)`: normally each build is done on the source tree, then the tree is cleaned with `mka clean`. If you want to be sure that each build is isolated from the others, set `BUILD_OVERLAY` to `true` (longer build time). Requires `--cap-add=SYS_ADMIN`.
* `LOCAL_MIRROR (false)`: change this to `true` if you want to create a local mirror of the LineageOS source (> 200 GB)
* `CRONTAB_TIME (now)`: instead of building immediately and exit, build at the specified time (uses standard cron format)
## Volumes
You also have to provide Docker some volumes, where it'll store the source, the
resulting builds, the cache and so on. The volumes are:
* `/srv/src`, for the LineageOS sources
* `/srv/zips`, for the output builds
* `/srv/logs`, for the output logs
* `/srv/ccache`, for the ccache
* `/srv/local_manifests`, for custom manifests (optional)
* `/srv/userscripts`, for the user scripts (optional)
When `SIGN_BUILDS` is `true`
* `/srv/keys`, for the signing keys
When `BUILD_OVERLAY` is `true`
* `/srv/tmp`, for temporary files
When `LOCAL_MIRROR` is `true`:
* `/srv/mirror`, for the LineageOS mirror
## Examples
### Build for bacon (officially supported), test keys, no patches
This example is the build script used for [LineageOS for microG](https://lineage.microg.org), which has integrated microG apps and F-Droid (with F-Droid Privileged Extension).
```
docker run \
--name=lineage-$(date +%Y%m%d_%H%M) \
--cap-add=SYS_ADMIN \
-d \
-e "USER_NAME=John Doe" \
-e "USER_MAIL=john.doe@awesome.email" \
-e "WITH_SU=false" \
-e "INCLUDE_PROPRIETARY=true" \
-e "RELEASE_TYPE=microG" \
-e "BRANCH_NAME=cm-13.0,cm-14.1" \
-e "DEVICE_LIST_CM_13_0=$DEVICES_CM13_0" \
-e "DEVICE_LIST_CM_14_1=$DEVICES_CM14_1" \
-e "OTA_URL=https://api.lineage.microg.org" \
-e "CRONTAB_TIME=now" \
-e "SIGNATURE_SPOOFING=restricted" \
-e "CUSTOM_PACKAGES=GmsCore GsfProxy FakeStore FDroid FDroidPrivilegedExtension MozillaNlpBackend NominatimNlpBackend com.google.android.maps.jar" \
-e "SIGN_BUILDS=true" \
-e "CLEAN_OUTDIR=false" \
-e "CLEAN_AFTER_BUILD=true" \
-e "ZIP_SUBDIR=true" \
-e "LOGS_SUBDIR=true" \
-e "DELETE_OLD_ZIPS=3" \
-e "DELETE_OLD_LOGS=3" \
-e "CCACHE_SIZE=540G" \
-v "/home/user/cache:/srv/ccache" \
-v "/home/user/mirror:/srv/mirror" \
-e "BRANCH_NAME=cm-14.1" \
-e "DEVICE_LIST=bacon" \
-v "/home/user/lineage:/srv/src" \
-v "/home/user/zips:/srv/zips" \
-v "/home/user/lineage_manifests:/srv/local_manifests" \
-v "/home/user/lineage_keys:/srv/keys" \
-v "/home/user/logs:/srv/logs" \
-v "/home/user/tmp:/srv/tmp" \
lineageos4microg/docker-lineage-cicd:multibranch
-v "/home/user/cache:/srv/ccache" \
lineageos4microg/docker-lineage-cicd
```
with the following XML in local_manifests
### Build for dumpling (officially supported), custom keys, restricted signature spoofing with integrated microG and FDroid
```
docker run \
-e "BRANCH_NAME=lineage-15.1" \
-e "DEVICE_LIST=dumpling" \
-e "SIGN_BUILDS=true" \
-e "SIGNATURE_SPOOFING=restricted" \
-e "CUSTOM_PACKAGES=GmsCore GsfProxy FakeStore MozillaNlpBackend NominatimNlpBackend com.google.android.maps.jar FDroid FDroidPrivilegedExtension " \
-v "/home/user/lineage:/srv/src" \
-v "/home/user/zips:/srv/zips" \
-v "/home/user/logs:/srv/logs" \
-v "/home/user/cache:/srv/ccache" \
-v "/home/user/keys:/srv/keys" \
-v "/home/user/manifests:/srv/local_manifests" \
lineageos4microg/docker-lineage-cicd
```
If there are already keys in `/home/user/keys` they will be used, otherwise a
new set will be generated before starting the build (and will be user for every
build).
The microG and FDroid packages are not present in the LineageOS repositories,
and must be provided through an XML in the `/home/user/manifests`. [This](https://github.com/lineageos4microg/android_prebuilts_prebuiltapks)
repo contains some of the most common packages for these kind of builds: to
include it create an XML (the name is irrelevant, as long as it ends with
`.xml`) in the `/home/user/manifests` folder with this content:
```
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<project path="prebuilts/prebuiltapks" name="lineageos4microg/android_prebuilts_prebuiltapks" remote="github" revision="master" />
<project name="lineageos4microg/android_prebuilts_prebuiltapks" path="prebuilts/prebuiltapks" remote="github" revision="master" />
</manifest>
```
The keys in /home/build/keys can be generated with
### Build for four devices on cm-14.1 and lineage-15.1 (officially supported), custom keys, restricted signature spoofing with integrated microG and FDroid, custom OTA server
```
subject='/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com'
mkdir keys
for x in releasekey platform shared media; do \
/home/user/source/development/tools/make_key keys/$x "$subject"; \
done
for c in cyngn{-priv,}-app testkey; do
for e in pk8 x509.pem; do
ln -s releasekey.$e keys/$c.$e;
done;
done
docker run \
-e "BRANCH_NAME=cm-14.1,lineage-15.1" \
-e "DEVICE_LIST_CM_14_1=onyx,thea" \
-e "DEVICE_LIST_LINEAGE_15_1=cheeseburger,dumpling" \
-e "SIGN_BUILDS=true" \
-e "SIGNATURE_SPOOFING=restricted" \
-e "CUSTOM_PACKAGES=GmsCore GsfProxy FakeStore MozillaNlpBackend NominatimNlpBackend com.google.android.maps.jar FDroid FDroidPrivilegedExtension " \
-e "OTA_URL=https://api.myserver.com/" \
-v "/home/user/lineage:/srv/src" \
-v "/home/user/zips:/srv/zips" \
-v "/home/user/logs:/srv/logs" \
-v "/home/user/cache:/srv/ccache" \
-v "/home/user/keys:/srv/keys" \
-v "/home/user/manifests:/srv/local_manifests" \
lineageos4microg/docker-lineage-cicd
```
### Build for a6000 (not officially supported), custom keys, restricted signature spoofing with integrated microG and FDroid
As there is no official support for this device, we first have to include the
sources in the source tree through an XML in the `/home/user/manifests` folder;
from [this](https://forum.xda-developers.com/lenovo-a6000/development/rom-lineageos-15-1-t3733747)
thread we get the links of:
* Device tree: [https://github.com/dev-harsh1998/android_device_lenovo_a6000](https://github.com/dev-harsh1998/android_device_lenovo_a6000)
* Common Tree: [https://github.com/dev-harsh1998/android_device_lenovo_msm8916-common](https://github.com/dev-harsh1998/android_device_lenovo_msm8916-common)
* Kernel: [https://github.com/dev-harsh1998/kernel_lenovo_msm8916](https://github.com/dev-harsh1998/kernel_lenovo_msm8916)
* Vendor blobs: [https://github.com/dev-harsh1998/proprietary-vendor_lenovo](https://github.com/dev-harsh1998/proprietary-vendor_lenovo)
Then, with the help of lineage.dependencies from the [device tree](https://github.com/dev-harsh1998/android_device_lenovo_a6000/blob/lineage-15.1/lineage.dependencies)
and the [common tree](https://github.com/dev-harsh1998/android_device_lenovo_msm8916-common/blob/lineage-15.1/lineage.dependencies),
we create an XML `/home/user/manifests/a6000.xml` with this content:
```
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<project name="dev-harsh1998/android_device_lenovo_a6000" path="device/lenovo/a6000" remote="github" />
<project name="dev-harsh1998/android_device_lenovo_msm8916-common" path="device/lenovo/msm8916-common" remote="github" />
<project name="dev-harsh1998/kernel_lenovo_msm8916" path="kernel/lenovo/a6000" remote="github" />
<project name="dev-harsh1998/proprietary-vendor_lenovo" path="vendor/lenovo" remote="github" />
<project name="LineageOS/android_device_qcom_common" path="device/qcom/common" remote="github" />
</manifest>
```
We also want to include our custom packages so, like before, create an XML (for
example `/home/user/manifests/custom_packages.xml`) with this content:
```
<?xml version="1.0" encoding="UTF-8"?>
<manifest>
<project name="lineageos4microg/android_prebuilts_prebuiltapks" path="prebuilts/prebuiltapks" remote="github" revision="master" />
</manifest>
```
We also set `INCLUDE_PROPRIETARY=false`, as the proprietary blobs are already
provided by the repo [https://github.com/dev-harsh1998/prorietary_vendor_lenovo](https://github.com/dev-harsh1998/prorietary_vendor_lenovo)
(so we don't have to include the TheMuppets repo).
Now we can just run the build like it was officially supported:
```
docker run \
-e "BRANCH_NAME=lineage-15.1" \
-e "DEVICE_LIST=a6000" \
-e "SIGN_BUILDS=true" \
-e "SIGNATURE_SPOOFING=restricted" \
-e "CUSTOM_PACKAGES=GmsCore GsfProxy FakeStore MozillaNlpBackend NominatimNlpBackend com.google.android.maps.jar FDroid FDroidPrivilegedExtension " \
-e "INCLUDE_PROPRIETARY=false" \
-v "/home/user/lineage:/srv/src" \
-v "/home/user/zips:/srv/zips" \
-v "/home/user/logs:/srv/logs" \
-v "/home/user/cache:/srv/ccache" \
-v "/home/user/keys:/srv/keys" \
-v "/home/user/manifests:/srv/local_manifests" \
lineageos4microg/docker-lineage-cicd
```