Thoughts dereferenced from the scratchpad noise. | Quick start guide to kas - best tool for setting up the Yocto projects

Introduction

If you are using the Yocto Project, you certainly have encountered the hassle of managing multiple layers and tracking their revisions.

I’ve been using the Yocto Project for nearly 3 years by now and have mostly been using the tool for this purpose. While I’m not a huge fan of it, it is relatively simple to use and gets the job of fetching layers and controlling their revisions done properly.

The only alternative I knew so far was the combo-layer, although it’s feature set was not enough for me to give up on repo and switch over.

Using kas with Yocto Project

According to the git history, the first public release of the project was at the Jun 14, 2017. I haven’t heard much about it. From my perspective, it gained some traction a couple of months ago - I’ve been seeing some mentions of the kas tool here and there on the various Yocto Project related mailing lists since then.

It seems to be a little bit more than just a tool for fetching and checking out a set of layers to given revisions. The feature set covers:

clone and checkout bitbake layers,
create default bitbake settings (machine, distro etc.),
launch minimal build environment, reducing the risk of host contamination (docker container),
initiate bitbake build process.

So far, we’ve mostly been doing the above with a mixture of the repo, our yocto-docker container and a set of shell scripts to automate things. It seems most of it can already be achieved using kas, as it has been developed specifically for managing and configuring the bitbake based projects.

Yocto BSP management comparison

Installation

According to the kas usage documentation, it can be installed natively via pip (python3 is required) or can be run inside docker container. I prefer the latter whenever possible, so I’m going to start with this one.

There are actually two containers available:

kasproject/kas - for standard Yocto Project builds,
kasproject/kas-isar - for isar builds.

Although in this case we are interested in the kasproject/kas container, I am happy to see the kasproject/kas-isar to be present, as we already have some use-cases for the isar project as well.

Although not mentioned in the usage documentation, it is advised to use the kas via the kas-docker script. For convenience, I’m placing it in my ~/bin directory, so it is available in the PATH:

1

ln -s /storage/projects/kas/kas-docker ~/bin/kas-docker

kas configuration

The kas file syntax and the project configuration process is nicely described int the kas project configuration documentation.

An even better way of understanding it may be to take a look at some real examples. I can advise taking a look at the kas.yml from the meta-iot2000.

Transition from repo manifest to kas file

As mentioned earlier, I’ve been using repo to manage the layers and some sample configuration files and a set of shell scripts to manage the build configurations. In the case of kas, all of it can be included in a single kas file.

Yocto layers in repo manifest

The repo manifest contains a list of layers to fetch and their revisions. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


    <project path="poky"
     name="poky"
     remote="yocto"
     revision="rocko"/>

    <project path="poky/meta-openembedded"
      name="openembedded/meta-openembedded"
      remote="github"
      revision="rocko"/>

    <project path="poky/meta-sunxi"
      name="linux-sunxi/meta-sunxi"
      remote="github"
      revision="rocko"/>

    <project path="poky/meta-rte"
      name="3mdeb_rte/meta-rte"
      remote="gitlab"
      revision="refs/tags/v0.4.1"/>

    <project path="poky/meta-swupdate"
     name="sbabic/meta-swupdate"
     remote="github"
     revision="rocko"/>

Additionally, we needed an example bblayers.conf file or some kind of shell script to enable the layers we need and adjust the paths to given build environment.

Yocto layers in kas file

The equivalent excerpt from the kas file would look like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


repos:
  # This repo contains the kas.yml file - there is no need to fetch it again.
  # It's current revision will be used to perform the build.
  meta-rte:

  poky:
    url: https://git.yoctoproject.org/git/poky
    refspec: 623b77885051174d0e05198843e739110977bd18
    layers:
      meta:
      meta-poky:
      meta-yocto-bsp:

  meta-sunxi:
    url: https://github.com/linux-sunxi/meta-sunxi
    refspec: 29b20da5e8cdea846c26d47a930d16114d71e0ca

  meta-openembedded:
    url: http://git.openembedded.org/meta-openembedded
    refspec: 8760facba1bceb299b3613b8955621ddaa3d4c3f
    layers:
      meta-oe:
      meta-python:
      meta-networking:

  meta-swupdate:
    url: https://github.com/sbabic/meta-swupdate
    refspec: f2d65d87485ada5a2d3a744fd7b9e46ec7e6b9f2

Note that it not only tells which repositories to fetch and which revisions to use. Based on the above information, kas will also automatically generate the bblayers.conf file with the required layers enabled there.

meta-rte Yocto layers

Transition from shell scripts to kas file

Build configuration in sample files and shell scripts

It is quite a common practice to ship some kind of sane, example build config file (i.e. local.conf file) when providing the Yocto BSP. Sometimes, when some conditional logic is necessary, shell scripts are being incorporated to modify the configuration files based on the user input.

Build configuration in kas file

All configurations can be maintained in a single kas file. In more complicated examples, it can be maintained in a set of kas files by using the include feature.

bblayers.conf

This will be at the top of the bblayers.conf file. Refer to the kas project configuration documentation for _header directive explanation.

1
2
3
4
5


bblayers_conf_header:
  standard: |
    POKY_BBLAYERS_CONF_VERSION = "2"
    BBPATH = "${TOPDIR}"
    BBFILES ?= ""

local.conf

This will be at the top of the local.conf file. Refer to the kas project configuration documentation for _header directive explanation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


local_conf_header:
  standard: |
    CONF_VERSION = "1"
    PACKAGE_CLASSES = "package_rpm"
    SDKMACHINE = "x86_64"
    USER_CLASSES = "buildstats image-mklibs image-prelink"
    PATCHRESOLVE = "noop"
  debug-tweaks: |
    EXTRA_IMAGE_FEATURES = "debug-tweaks"
  diskmon: |
    BB_DISKMON_DIRS = "\
        STOPTASKS,${TMPDIR},1G,100K \
        STOPTASKS,${DL_DIR},1G,100K \
        STOPTASKS,${SSTATE_DIR},1G,100K \
        STOPTASKS,/tmp,100M,100K \
        ABORT,${TMPDIR},100M,1K \
        ABORT,${DL_DIR},100M,1K \
        ABORT,${SSTATE_DIR},100M,1K \AC
        ABORT,/tmp,10M,1K"

MACHINE and DISTRO

As you probably know, it is essential to set the set the MACHINE and DISTRO. In kas file it is as simple as that:

1
2


machine: orange-pi-zero
distro: rte

target

This is the default recipe which will be built during the kas build action. Usually, it will be the main image of our BSP. For example:

1

target: core-image-minimal

Usage

It seems that when using docker container, we have two modes of operation:

build the target image with a single command,
enter system shell and work directly from there.

Shell

This mode can certainly be useful for debugging (both issues with our Yocto build and kas configuration).

Below command will make sure that the repositories specified in the kas file are properly fetched and checked out. Then it will give us access to the container’s system shell.

1

kas-docker shell meta-rte/kas.yml

I’m using the zsh and faced below error:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


2019-01-11 14:49:51 - ERROR    - [Errno 2] No such file or directory: '/usr/bin/zsh'
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/kas/kas.py", line 161, in main
    sys.exit(kas(sys.argv[1:]))
  File "/usr/local/lib/python3.5/dist-packages/kas/kas.py", line 149, in kas
    if plugin().run(args):
  File "/usr/local/lib/python3.5/dist-packages/kas/shell.py", line 112, in run
    macro.run(ctx, args.skip)
  File "/usr/local/lib/python3.5/dist-packages/kas/libcmds.py", line 63, in run
    command.execute(ctx)
  File "/usr/local/lib/python3.5/dist-packages/kas/shell.py", line 137, in execute
    cwd=ctx.build_dir)
  File "/usr/lib/python3.5/subprocess.py", line 247, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/lib/python3.5/subprocess.py", line 676, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1282, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/zsh'

It seems that the kas-docker script respects our SHELL environment variable. This can be easily overridden with:

1

SHELL=bash kas-docker shell meta-rte/kas.yml

Build Yocto image with kas

The default build can be performed with below command. It will fetch the required layers, make sure they have desired revisions, modify the configuration files accordingly and execute the bitbake task to build the recipe specified by the target reference in the kas configuration file.

1

kas-docker build meta-rte/kas.yml

The build output log looks like:

Note the warning at the top of the log output. Despite using the aufs as the storage driver for docker, the wic image built well on my setup. The explanation of the warning can be found in this kas commit

My host setup was:

1
2
3


OS: Ubuntu 16.04.1
kernel: 4.15.0-43-generic
docker: 18.09.0

Private repositories

When fetching from private repositories is needed (either during the layers fetching or during the build process itself), we need to expose access keys (usually SSH keys) somehow. It seems that the preferred way (at least when using the kas-docker) is to use the -ssh-dir switch of the script:

1

kas-docker --ssh-dir ~/ssh-keys shell meta-rte/kas.yml

The contents of the ~/ssh-keys can look like:

1
2
3
4
5


config
github_key_ro
github_key_ro.pub
gitlab_key_ro
gitlab_key_ro.pub

And the ~/ssh-keys/config file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


Host gitlab.com
    HostName       gitlab.com
    User           git
    IdentityFile   ~/.ssh/gitlab_key_ro
    StrictHostKeyChecking no
    IdentitiesOnly yes

Host github.com
    HostName       github.com
    User           git
    IdentityFile   ~/.ssh/github_key_ro
    StrictHostKeyChecking no
    IdentitiesOnly yes

Final example

The final kas file for meta-rte can be found in the meta-rte repository. The documentation on how to build the system for the rte using kas can be found in the meta-rte README.

Conclusion

After some initial work with the kas, it seems like a great tool for managing bitbake based BSP. It seems that it is capable of replacing most of our legacy way of managing bitbake layers and configuration.

In my opinion, the kas project definitely deserves some more popularity. At the moment it has less than 30 stars on github. I can’t wait to see how it would fit in some more complex use-cases we have.

References

Maciej Pijanowski

Engineering Manager at 3mdeb with years of experience in engineering and management. Open-source software enthusiast and contributor. Interested in embedded systems in general, build systems, security.

Quick start guide to kas - best tool for setting up the Yocto projects