Introduction
If you are using the Yocto Project, you certainly have encountered the hassle of managing multiple layers and tracking their revisions.
I’ve been using the Yocto Project for nearly 3 years by now and have mostly been using the tool for this purpose. While I’m not a huge fan of it, it is relatively simple to use and gets the job of fetching layers and controlling their revisions done properly.
The only alternative I knew so far was the combo-layer, although it’s feature set was not enough for me to give up on repo and switch over.
Using kas with Yocto Project
According to the git history, the first public release of the project was at the Jun 14, 2017. I haven’t heard much about it. From my perspective, it gained some traction a couple of months ago - I’ve been seeing some mentions of the kas tool here and there on the various Yocto Project related mailing lists since then.
It seems to be a little bit more than just a tool for fetching and checking out a set of layers to given revisions. The feature set covers:
- clone and checkout
bitbake
layers, - create default
bitbake
settings (machine, distro etc.), - launch minimal build environment, reducing the risk of host contamination
(
docker
container), - initiate
bitbake
build process.
So far, we’ve mostly been doing the above with a mixture of the repo, our
yocto-docker container and a set of shell scripts to automate things. It
seems most of it can already be achieved using kas, as it has been developed
specifically for managing and configuring the bitbake
based projects.
Installation
According to the kas usage documentation, it can be installed natively via
pip
(python3
is required) or can be run inside docker
container. I prefer
the latter whenever possible, so I’m going to start with this one.
There are actually two containers available:
- kasproject/kas - for standard Yocto Project builds,
- kasproject/kas-isar - for isar builds.
Although in this case we are interested in the kasproject/kas container, I am happy to see the kasproject/kas-isar to be present, as we already have some use-cases for the isar project as well.
Although not mentioned in the
usage documentation,
it is advised
to use the kas
via the
kas-docker script.
For convenience, I’m placing it in my ~/bin
directory, so it is available in
the PATH
:
1
|
ln -s /storage/projects/kas/kas-docker ~/bin/kas-docker |
kas configuration
The kas
file syntax and the project configuration process is nicely described
int the
kas project configuration documentation.
An even better way of understanding it may be to take a look at some real examples. I can advise taking a look at the kas.yml from the meta-iot2000.
Transition from repo manifest to kas file
As mentioned earlier, I’ve been using repo to manage the layers and some
sample configuration files and a set of shell scripts to manage the build
configurations. In the case of kas, all of it can be included in a single
kas
file.
Yocto layers in repo manifest
The repo manifest contains a list of layers to fetch and their revisions. For example:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
<project path="poky" name="poky" remote="yocto" revision="rocko"/> <project path="poky/meta-openembedded" name="openembedded/meta-openembedded" remote="github" revision="rocko"/> <project path="poky/meta-sunxi" name="linux-sunxi/meta-sunxi" remote="github" revision="rocko"/> <project path="poky/meta-rte" name="3mdeb_rte/meta-rte" remote="gitlab" revision="refs/tags/v0.4.1"/> <project path="poky/meta-swupdate" name="sbabic/meta-swupdate" remote="github" revision="rocko"/> |
Additionally, we needed an example bblayers.conf
file or some kind of shell
script to enable the layers we need and adjust the paths to given build
environment.
Yocto layers in kas file
The equivalent excerpt from the kas
file would look like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
repos: # This repo contains the kas.yml file - there is no need to fetch it again. # It's current revision will be used to perform the build. meta-rte: poky: url: https://git.yoctoproject.org/git/poky refspec: 623b77885051174d0e05198843e739110977bd18 layers: meta: meta-poky: meta-yocto-bsp: meta-sunxi: url: https://github.com/linux-sunxi/meta-sunxi refspec: 29b20da5e8cdea846c26d47a930d16114d71e0ca meta-openembedded: url: http://git.openembedded.org/meta-openembedded refspec: 8760facba1bceb299b3613b8955621ddaa3d4c3f layers: meta-oe: meta-python: meta-networking: meta-swupdate: url: https://github.com/sbabic/meta-swupdate refspec: f2d65d87485ada5a2d3a744fd7b9e46ec7e6b9f2 |
Note that it not only tells which repositories to fetch and which revisions to
use. Based on the above information, kas
will also automatically generate the
[bblayers.conf]() file with the required layers enabled there.
Transition from shell scripts to kas file
Build configuration in sample files and shell scripts
It is quite a common practice to ship some kind of sane, example build config
file (i.e. [local.conf]() file) when providing the Yocto BSP
. Sometimes,
when some conditional logic is necessary, shell scripts are being incorporated
to modify the configuration files based on the user input.
Build configuration in kas file
All configurations can be maintained in a single kas
file. In more complicated
examples, it can be maintained in a set of kas
files by using the
include feature.
bblayers.conf
This will be at the top of the bblayers.conf file. Refer to the
kas project configuration documentation for _header
directive explanation.
1 2 3 4 5 |
bblayers_conf_header: standard: | POKY_BBLAYERS_CONF_VERSION = "2" BBPATH = "${TOPDIR}" BBFILES ?= "" |
local.conf
This will be at the top of the local.conf file. Refer to the
kas project configuration documentation for _header
directive explanation.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
local_conf_header: standard: | CONF_VERSION = "1" PACKAGE_CLASSES = "package_rpm" SDKMACHINE = "x86_64" USER_CLASSES = "buildstats image-mklibs image-prelink" PATCHRESOLVE = "noop" debug-tweaks: | EXTRA_IMAGE_FEATURES = "debug-tweaks" diskmon: | BB_DISKMON_DIRS = "\ STOPTASKS,${TMPDIR},1G,100K \ STOPTASKS,${DL_DIR},1G,100K \ STOPTASKS,${SSTATE_DIR},1G,100K \ STOPTASKS,/tmp,100M,100K \ ABORT,${TMPDIR},100M,1K \ ABORT,${DL_DIR},100M,1K \ ABORT,${SSTATE_DIR},100M,1K \AC ABORT,/tmp,10M,1K" |
MACHINE and DISTRO
As you probably know, it is essential to set the set the
MACHINE
and
DISTRO.
In kas
file it is as simple as that:
1 2 |
machine: orange-pi-zero distro: rte |
target
This is the default recipe which will be built during the kas build
action.
Usually, it will be the main image of our BSP
. For example:
1
|
target: core-image-minimal |
Usage
It seems that when using docker
container, we have two modes of operation:
- build the target image with a single command,
- enter system shell and work directly from there.
Shell
This mode can certainly be useful for debugging (both issues with our Yocto
build and kas
configuration).
Below command will make sure that the repositories specified in the kas
file
are properly fetched and checked out. Then it will give us access to the
container’s system shell.
1
|
kas-docker shell meta-rte/kas.yml |
I’m using the zsh
and faced below error:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
2019-01-11 14:49:51 - ERROR - [Errno 2] No such file or directory: '/usr/bin/zsh' Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/kas/kas.py", line 161, in main sys.exit(kas(sys.argv[1:])) File "/usr/local/lib/python3.5/dist-packages/kas/kas.py", line 149, in kas if plugin().run(args): File "/usr/local/lib/python3.5/dist-packages/kas/shell.py", line 112, in run macro.run(ctx, args.skip) File "/usr/local/lib/python3.5/dist-packages/kas/libcmds.py", line 63, in run command.execute(ctx) File "/usr/local/lib/python3.5/dist-packages/kas/shell.py", line 137, in execute cwd=ctx.build_dir) File "/usr/lib/python3.5/subprocess.py", line 247, in call with Popen(*popenargs, **kwargs) as p: File "/usr/lib/python3.5/subprocess.py", line 676, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.5/subprocess.py", line 1282, in _execute_child raise child_exception_type(errno_num, err_msg) FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/zsh' |
It seems that the kas-docker
script respects our SHELL
environment
variable. This can be easily overridden with:
1
|
SHELL=bash kas-docker shell meta-rte/kas.yml |
Build Yocto image with kas
The default build can be performed with below command. It will fetch the
required layers, make sure they have desired revisions, modify the configuration
files accordingly and execute the bitbake
task to build the recipe specified
by the target
reference in the kas
configuration file.
1
|
kas-docker build meta-rte/kas.yml |
The build output log looks like:
Note the warning at the top of the log output. Despite using the aufs
as the
storage driver for docker
, the wic
image built well on my setup. The
explanation of the warning can be found in
this kas commit
My host setup was:
1 2 3 |
OS: Ubuntu 16.04.1 kernel: 4.15.0-43-generic docker: 18.09.0 |
Private repositories
When fetching from private repositories is needed (either during the layers
fetching or during the build process itself), we need to expose access keys
(usually SSH
keys) somehow. It seems that the preferred way (at least when
using the kas-docker
) is to use the -ssh-dir
switch of the script:
1
|
kas-docker --ssh-dir ~/ssh-keys shell meta-rte/kas.yml |
The contents of the ~/ssh-keys
can look like:
1 2 3 4 5 |
config github_key_ro github_key_ro.pub gitlab_key_ro gitlab_key_ro.pub |
And the ~/ssh-keys/config
file:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
Host gitlab.com HostName gitlab.com User git IdentityFile ~/.ssh/gitlab_key_ro StrictHostKeyChecking no IdentitiesOnly yes Host github.com HostName github.com User git IdentityFile ~/.ssh/github_key_ro StrictHostKeyChecking no IdentitiesOnly yes |
Final example
The final kas
file for meta-rte
can be found in the meta-rte
repository. The documentation on how to
build the system for the rte using kas
can be found
in the meta-rte README.
Conclusion
After some initial work with the kas, it seems like a great tool for managing
bitbake
based BSP
. It seems that it is capable of replacing most of our
legacy way of managing bitbake
layers and configuration.
In my opinion, the kas project definitely deserves some more popularity. At the moment it has less than 30 stars on github. I can’t wait to see how it would fit in some more complex use-cases we have.
References
