Building and flashing a secured AOSP build with verified boot and separate lockscreen password for the Nexus 5X

Disclaimer and License

All data and information provided in this tutorial is for informational purposes only. The author makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information on this tutorial and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis.

In no event, the author or howtoforge will be liable for any loss or damage including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising from loss of data or profits arising out of, or in connection with, the use of this tutorial.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License, and code samples are licensed under the Apache 2.0 License.

Motivation

The Nexus 5X and 6P were the first devices that supported verified boot based on user-supplied (and not vendor-enforced) signing keys. Before its “implosion” [1], CopperheadOS (a security-enhanced AOSP variant) used to provide good documenation and scripts for building and flashing a secure AOSP version [2]. However, the project stopped providing any updates in the recent months, so most former users are looking for viable alternatives.

My impression is that many users agree that running a self-built AOSP ROM is a much better alternative to other options such as switching to e.g. LineageOS. There are a number of good reasons for this:

• AOSP is plain stock and has only few possibly unwanted features
• AOSP can be built as “user” instead of “userdebug” build vartiant and, thus, is expected to be more secure (I tried compiling user builds of LineageOS, but they seem to be broken due to the invasive changes LineageOS made to the AOSP sources)
• Once sources are fetched, AOSP can be simply built. Unlike LineageOS, it does not start downloading additional sources during the build.

Since the Nexus 5X is believed to be a well-supported developer device, I expected the process of building AOSP to be easy and well documented. However, it turns out that there are a number of caveats:

• including the vendor binaries by following the official documentation results in incomplete builds that cannot be used for incremental updates without unlocking and wiping each time (the vendor partition, radio ROM etc. is not included in the build)
• the official documentation does not describe how to use verified build in a way that is directly applicable. There is better documentation in the CopperheadOS docs [05], but the instructions rely on outdated scripts that are not applicable for AOSP.
• there is no documentation on how to use a “weak” PIN as passphrase but a strong password as disk encryption key (unlike newer Pixel devices, the Nexus 5X is based on the older FDE approach). The methods that work for LineageOS devices are not applicable as they assume the device is rooted which is not the case for regular AOSP user builds.

This tutorial aims to provide detailed instructions on how to solve these caveats, building and flashing AOSP for the Nexus 5X with verified boot and using separate lockscreen/encryption secrets. It should also apply for the Nexus 6P with small changes, but I was unable to test it since I didn’t have a Nexus 6P at hand.

Except for a small script collection (required for properly extracting the vendor blobs from Binaries supplied by Google) and its dependency “oatdump” (that is downloaded as binary from on a public share) the instructions do not make use of any “unofficial” (in the sense of “non-Google-provided”) third-party resources.

Be aware of the following freedom issues:

• The AOSP source tree contains a number of prebuilt binaries (e.g. toolchain, Linux kernel, …). While these binaries could be rebuilt from source, the needed steps are not covered in this tutorial.
• The source code for the vendor blobs required for using many hardware components of the Nexus 5X is not publically available!
• The tool “android-prepare-vendor” used to extract the proprietary vendor files uses prebuilt binaries itself (some even externally hosted).
  

Requirements and assumptions

The tutorial assumes that you have the following prerequisites (other versions/distributions might work as well but could require other or additional packages):

– a Nexus 5X with an unlocked bootloader (unlocking is not covered in this tutorial)

– (virtual) machine running Debian9 in the x86_64 variant, used exclusively for our purpose (we assume you use sudo, if not, adapt the commands)

– at least 5 GB of RAM (more is better)

– aprox 200 MB of disk space

– fast internet connection (we need to download around 30G of data)

Installing dependencies

First, install dependencies as described in the LineageOS build instructions [3] (AOSP build instructions do not provide this list):

sudo apt install bc bison build-essential ccache curl flex g++-multilib gcc-multilib git gnupg gperf imagemagick lib32ncurses5-dev lib32readline-dev lib32z1-dev liblz4-tool libncurses5-dev libsdl1.2-dev libssl-dev libwxgtk3.0-dev libxml2 libxml2-utils lzop pngcrush rsync schedtool squashfs-tools xsltproc zip zlib1g-dev

Now, install additional dependencies:

sudo apt install cmake zip unzip openjdk-8-jdk-headless

Then, setup a bin path in your home directory (in Debian 9 this path is automatically configured in the bash profile):

mkdir -p ~/bin

Install the repo command:

curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo

Verify the repo binary’s checksum. It should be e147f0392686c40cfd7d5e6f332c6ee74c4eab4d24e2694b3b0a0c037bf51dc5 for the current version 1.23. for later versions check the AOSP build instructions page [4]. Use the following command to compute the checksum:

sha256sum ~/bin/repo

Next, provide a git identity by running the following commands (you can leave the example data in if you prefer to remain anonymous):

git config –global user.email “[email protected]“

git config –global user.name “Your Name”

Unfortunately, the brotli package (required to pack the builds) in Debian 9 is too old, so we need to build the current version ourselves. First get the source code and change in its directory:

git clone https://github.com/google/brotli.git

Run the build (replace -j15 by the number of your cpu threads):

cd ~/brotli

./configure-cmake

make -j15

Finally, copy the resulting binary to our bin path:

cp brotli ~/bin/

Finally, logout and login again so your bash profile is re-read.

Getting the vendor blobs

There are several issues with using the vendors blobs from the binary driver packages provided by Google (see [5]). To solve them, we use the set of external scripts “android-prepare-vendor” by “anestisb” that extracts the vendor blobs them from the factory images instead.

First, clone the repository:

git clone https://github.com/anestisb/android-prepare-vendor.git

Using Google’s site [6], find out the latest Build tag for your Nexus 5X (currently it is OPM6.171019.030.K1).

Change to the repository, create an output directory and run the script (we run the script as root due to issues with fuse in Debian 9):

cd android-prepare-vendor

mkdir bullhead-blobs

sudo ./execute-all.sh -k -d bullhead -a bullhead -b OPM6.171019.030.K1 -o bullhead-blobs

Downloading AOSP sources

Note: The following steps lack instructions for verifying the downloaded source.

First, create a directory where the sources will be stored:

mkdir -p ~/aosp

Knowing the current build tag for the Nexus 5X, find out what the corresponding Android tag is using the overview available at [6]. Then, checkout the Android manifest for the corresponding branch (in this example, we use android-8.1.0_r46):

cd ~/aosp

repo init -u https://android.googlesource.com/platform/manifest -b android-8.1.0_r46

Now, you can edit the ~/aosp.repo/manifest.xml to exclude certain repositories or to include others (note: this is purely optional). Use <!– and –> as usual in XML to comment out repositories. I recommend excluding/replacing the following:

• <project path=”packages/apps/QuickSearchBox” name=”platform/packages/apps/QuickSearchBox” groups=”pdk-fs” /> – the QuickSearchBox is mostly broken in AOSP anyways
• <project path=”device/lge/bullhead” name=”device/lge/bullhead” groups=”device,bullhead,pdk” /> – replace this with a repository that contains a patched device tree where the two faulty cpu cores are disabled. This comes with a ~30% performance penalty. Yet, recommended, even if you have Nexus 5X that is not affected by the bootloop, as it is likely it will be affected in the future.
• <project path=”packages/apps/Calendar” name=”platform/packages/apps/Calendar” groups=”pdk-fs” /> – there are better alternatives to the stock AOSP calendar you can install later (such as Etar)
• <project path=”packages/apps/Messaging” name=”platform/packages/apps/Messaging” groups=”pdk-fs” /> – Silence.im is a better alternative for the AOSP messenging app
• <project path=”packages/apps/Camera2″ name=”platform/packages/apps/Camera2″ groups=”pdk-fs” /> – OpenCamera is a better alternative for the stock camera

Now, fetch all the repositores (can take a long time, mostly depending on your internet connection):

repo sync

Finally, copy the previously generated vendor blobs as root (this is required, or qmus and other blobs will be missing and cause later compilation failure) into the vendor directory of your AOSP tree (replace the factory build number by the current one):

sudo cp -av ~/android-prepare-vendor/bullhead-blobs/bullhead/opm6.171019.030.k1/vendor .

sudo cp -av ~/android-prepare-vendor/bullhead-blobs/bullhead/opm6.171019.030.k1/vendor_overlay .

Make your user the owner of the vendor directories (or the build will fail later). Replace yourusername with your actual username:

sudo chown -R yourusername:yourusername ~/aosp/vendor

sudo chown -R yourusername:yourusername ~/aosp/vendor_overlay

Generating Keys

Set the build variables:

source build/envsetup.sh

Build the tool needed for generating the verity key:

make generate_verity_key

Create a directory for storing your keys (CopperheadOS docs [2] recommend using a separate key for each device, in this case bullhead):

mkdir -p keys/bullhead

Now it's time to generate the keys (do not set passwords on your keys):

cd keys/bullhead

../../development/tools/make_key releasekey '/C=DE/ST=Hometown/L=XX/O=yournamehere/OU=yournamehere/CN=yournamehere/[email protected]'

../../development/tools/make_key platform '/C=DE/ST=Hometown/L=XX/O=yournamehere/OU=yournamehere/CN=yournamehere/[email protected]'

../../development/tools/make_key shared '/C=DE/ST=Hometown/L=XX/O=yournamehere/OU=yournamehere/CN=yournamehere/[email protected]'

../../development/tools/make_key media '/C=DE/ST=Hometown/L=XX/O=yournameher /OU=yournamehere/CN=yournamehere/[email protected]'

../../development/tools/make_key verity '/C=DE/ST=Hometown/L=XX/O=yournamehere/OU=yournamehere/CN=yournamehere/[email protected]'

cd ~/aosp


Convert the verity key to the format required by AOSP:

out/host/linux-x86/bin/generate_verity_key -convert keys/bullhead/verity.x509.pem keys/bullhead/verity_key

Compiling

Make sure your build variables are set:

cd ~/aosp

source build/envsetup.sh

Create a user-lunchconfig for the bullhead device (replace user by userdebug if you want a userdebug configuration instead):

lunch aosp_bullhead-user

Disable Jack (it often causes compile trouble and has been deprecated in Android 9 anyways):

export ANDROID_COMPILE_WITH_JACK=false

Compile the target-files-package (replace -j15 by the number of your cpu threads):

make target-files-package -j15

Packaging and signing

Create a directory to store the output files (outside of the usual build directory named out):

mkdir dist

Run the dist target (replace -j15 by the number of your cpu threads):

make dist -j15

Create a signed-target-files package, replacing the default test keys with your keys (replace yourusername with your actual username on the system):

build/tools/releasetools/sign_target_files_apks -o -d keys/bullhead –replace_verity_public_key keys/bullhead/verity_key.pub –replace_verity_private_key keys/bullhead/verity –replace_verity_keyid keys/bullhead/verity.x509.pem out/dist/aosp_bullhead-target_files-eng.yourusername.zip dist/signed-target-files.zip

Create a signed OTA package:

build/tools/releasetools/ota_from_target_files -k keys/bullhead/releasekey dist/signed-target-files.zip dist/signed-ota-update.zip

Flashing

Boot your Nexus 5X device into bootloader (hold volume down, then hit power).

Connect your device via USB to your machine (and make it available to the VM in case you build in a VM). You can also copy the contents from the dist directory to another machine and flash from there, but we assume you flash from using the fastboot/adb binaries built from AOSP sources (if you flash from outside, make sure your fastboot binary is recent).

Unpack the images from the signed-target-files.zip:

cd ~/aosp/dist

unzip signed-target-files.zip IMAGES/*


Now, flash all the images:

../out/host/linux-x86/bin/fastboot flash boot boot.img

../out/host/linux-x86/bin/fastboot flash recovery recovery.img

../out/host/linux-x86/bin/fastboot flash vendor vendor.img

../out/host/linux-x86/bin/fastboot flash system system.img

Select “reboot system” using your device’s physical buttons and make sure your new system works.

Finally, reboot back to the bootloader, and re-lock it (will erase all data):

out/host/linux-x86/bin/fastboot flash oem locking

That’s it!

Setting separate boot/lockscreen secrets

Setting a separate boot/lockscreen password can be done with a small trick:

• Unlock the bootloader (wipes all data)
• compile and flash a userdebug build (see above)
• lock the bootloader
• set a lockscreen pin/password using the Android UI. Make sure you choose the right one because you won’t be able to change it again without wiping your data once you switch to the user build.
• connect to the device via adb
• as root, run the following command: vdc cryptfs changepw password your-new-password
• reboot, and make sure it works
• compile a regular user build (do not unlock the bootloader!)
• flash the user build

Note: Without additional steps, the recovery does not allow to flash older builds. Thus, you need to flash a user build that is newer than your userdebug build!

Handling updates

If you want to upgrade to a newer AOSP release, you first need to find out the new release number.

Then, wipe the old manifest (make a backup, in case you made changes you want to redo on the updated one):

cd ~/aosp

rm -rf .repo/manifests.git

rm -rf .repo/manifest.xml

Also, wipe the (then outdated) vendor blobs and the prepare-vendor repository:

rm -rf ~/android-prepare-vendor

rm -rf vendor

rm -rf vendor_overlay

Also, clean the build tree and the build artifacts:

rm -rf out

rm -rf dist

Then, redo ONLY the following steps:

• Getting the vendor blobs
• Downloading AOSP sources (this will be much quicker, because only the changes will be pulled in)
• Compiling
• Packaging and signing

In theory, you should be able to sideload updates as new OTA packages from recovery without wiping (since it will be signed signed with the same – your – keys). In practice, it does not work yet (see the next section). There, you will also have to proceed as follows:

• Backup all your data
• Unlock the bootloader (your data will be wiped)
• Flash the updated images and re-lock your bootloader as described in the section “Flashing”

WIP: Signed OTA updates

This section is WIP, the described instructions do not work yet!

In theory, it should be possible to create and flash signed OTA updates from recovery. However, all my attempts to do this resulted in a “Signature verification failed” error. Since this works when using the Google-supplied vendor files directly instead of using android-prepare-vendor, I assume it is related to the vendor files or other files (like the bootloader or radio images) not being properly signed.

Create the signed OTA package as follows:

build/tools/releasetools/ota_from_target_files -k keys/bullhead/releasekey dist/signed-target-files.zip dist/signed-ota-update.zip

Reboot to recovery using your device’s physical buttons.

In recovery,you will see a small android symbol. Hold down the power button and press volume up to get into the recovery menu.

Now, select “update from adb” using your device’s physical buttons.

Sideload your signed OTA package:

out/host/linux-x86/bin/adb sideload dist/signed-ota-update.zip

References

[1] https://www.reddit.com/r/CopperheadOS/comments/8qdnn3/goodbye/

[2] https://copperhead.co/android/docs/building

[3] https://wiki.lineageos.org/devices/bullhead/build

[4] https://source.android.com/setup/build/downloading

[5] https://github.com/anestisb/android-prepare-vendor

[6] https://source.android.com/setup/start/build-numbers.html#source-code-tags-and-builds