Commit Graph

29 Commits

Author SHA1 Message Date
Martin Roth 0bb9dcce19 util/docker: Switch back to root user in jenkins-node
Leaving the user as coreboot caused the entrypoint to run as coreboot,
which means we couldn't mount directories or run sshd correctly.

Switching to root at the end of the file fixes this.

Signed-off-by: Martin Roth <gaumless@gmail.com>
Change-Id: Ie10e1d7ad4def0faafe3bcd580a77e23c3bfe948
Reviewed-on: https://review.coreboot.org/c/coreboot/+/79067
Reviewed-by: Felix Singer <service+coreboot-gerrit@felixsinger.de>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2023-11-16 23:18:47 +00:00
Martin Roth 162b6484ff util/docker/jenkins-node: Install pip modules into virtual env
Instead of installing the pip modules system-wide, and possibly causing
conflicts, install them into a virtual environment for the coreboot
user.

If we wanted to, in the future, we could install different versions of
the modules into different virtual environment directories to allow
for testing or anything else we needed.

Change-Id: I49c749a13a698bfb7af29bf07e42ac14b67b2ae7
Signed-off-by: Martin Roth <gaumless@gmail.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/79006
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Felix Singer <service+coreboot-gerrit@felixsinger.de>
2023-11-12 04:53:44 +00:00
Martin Roth dbc33f9376 util/docker: change main branch back to master for encapsulate
The branch for the encapsulate tool accidentally got caught up in the
switch from master to main. The default branch for this tool has not
changed, so still needs to be referenced as master.

Change-Id: I0ff47308dcbf30888e4e88637bab63f20467307a
Signed-off-by: Martin Roth <gaumless@gmail.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/79005
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Felix Singer <service+coreboot-gerrit@felixsinger.de>
2023-11-12 04:53:37 +00:00
Stefan Reinauer 38d8a6a570 Switch jenkins node over to use encapsulate main branch
This is only needed once we want to recreate the docker

Change-Id: I493acb4de615508b08826f814ef6ac1b37cbdf0c
Signed-off-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/75781
Reviewed-by: Martin L Roth <gaumless@gmail.com>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2023-09-11 06:57:26 +00:00
Nicholas Chin 2fe63a1ad6 util/docker: Update Dockerfiles for building the documentation
The doc.coreboot.org container is several years out of date, using the
three year old Alpine 3.8 as the base image along with Sphinx related
pip packages which are even older. Accordingly, update the documentation
related pip packages in the coreboot-jenkins-node container as well.

- Update doc.coreboot.org to Alpine 3.18.3
- Update documentation related pip packages on coreboot-jenkins-node
  and doc.coreboot.org to the latest versions on PyPI
- Update Sphinx to 6.2.1 as the latest version of sphinx_rtd_theme does
  not yet support sphinx >= 7

The updates also noticeably improve performance, dropping documentation
build times from ~75 s down to ~42 s on my system from the Alpine+Python
updates alone, and further down to ~35 s with the rest of the updates.

TEST: The documentation builds and renders properly when built using the
updated container.

Change-Id: I38dfd22ee71c3779ab5fd3b3060e4675e9e3fe54
Signed-off-by: Nicholas Chin <nic.c3.14@gmail.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/73159
Reviewed-by: Felix Singer <service+coreboot-gerrit@felixsinger.de>
Reviewed-by: Martin L Roth <gaumless@gmail.com>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2023-08-21 22:39:06 +00:00
Martin Roth 1e193d01ea util/docker/jenkins-node: Don't install python modules as root
When installing the python modules with pip3 as root, the installer
throws a lot of warnings about conflicts and recommends that it not
be run that way.  This change installs the python modules as the
coreboot user instead. The --break-system-packages argument can now
be removed.

It takes along some other changes made to the coreboot home directory
which also don't need to be run as root, and now adds the .local/bin
directory into the path.

The trailing docker PATH configuration is discarded as cleanup - it
doesn't have any effect.  Nothing uses it in the Dockerfile, and it
doesn't end up updating the path, which is set by /etc/profile.

Change-Id: Ie8273009bb527e267584bba84504191aa7294ca3
Signed-off-by: Martin Roth <gaumless@gmail.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/76855
Reviewed-by: Felix Singer <service+coreboot-gerrit@felixsinger.de>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2023-08-21 17:05:29 +00:00
Elyes Haouas 9f59d2a906 coreboot-jenkins-node/Dockerfile: Upgrade lua5 from lua5.3 to lua5.4
Change-Id: Ic1450f0fa8eb69273aa907dea2eba8f7e7131ef1
Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/72510
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Felix Singer <service+coreboot-gerrit@felixsinger.de>
2023-08-04 20:55:45 +00:00
Felix Singer a87da91719 util/docker/jenkins-node: Drop Zephyr SDK
The version of the Zephyr SDK that is used is quite old and Zephyr
hasn't been really used. Thus, drop it and also its dependencies from
u-boot.

Change-Id: Ie498d687e1566133adf650166117d8f68fcfdaf6
Signed-off-by: Felix Singer <felixsinger@posteo.net>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/73979
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Matt DeVillier <matt.devillier@gmail.com>
Reviewed-by: Patrick Georgi <patrick@coreboot.org>
2023-04-25 05:36:13 +00:00
Felix Singer fa41fef557 util/docker/jenkins-node: Allow pip to install packages system-wide
Call pip3 with `--break-system-packages` allowing it to install packages
system-wide. This fixes building the Docker container.

Change-Id: Id093f2c69fec43556c434fbca7b36095a7e6bd97
Signed-off-by: Felix Singer <felixsinger@posteo.net>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/74292
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Matt DeVillier <matt.devillier@gmail.com>
Reviewed-by: Martin L Roth <gaumless@gmail.com>
2023-04-25 05:36:03 +00:00
Felix Singer 98d7157a6d util/docker/jenkins-node: Merge package installations into first step
It's not necessary to have multiple steps for installing packages and
requirements. Just merge the two install steps to one.

Change-Id: Ibe620e5b20a5f1a5d4e1c4c98942c136f450f280
Signed-off-by: Felix Singer <felixsinger@posteo.net>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/74245
Reviewed-by: Matt DeVillier <matt.devillier@gmail.com>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Martin L Roth <gaumless@gmail.com>
2023-04-25 05:35:57 +00:00
Martin Roth 70191da272 util/docker: Update setuptools to get dtc building
python3-setuptools installed via apt-get is not currently working to
build dtc from git.kernel.org.  Falling back to setuptools version
58.2.0 allows it to build again.

The failure message was:
/usr/lib/python3/dist-packages/setuptools/installer.py:27:
SetuptoolsDeprecationWarning: setuptools.installer is deprecated.
Requirements should be satisfied by a PEP 517 installer.

Signed-off-by: Martin Roth <gaumless@gmail.com>
Change-Id: I04cb6d776c3748f9a4b0cfc4ffd4f46458560d3d
Reviewed-on: https://review.coreboot.org/c/coreboot/+/71500
Reviewed-by: Felix Singer <felixsinger@posteo.net>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-12-25 13:22:13 +00:00
Felix Singer b45b48de73 util/docker/coreboot-jenkins-node: Install cmocka
flashrom uses cmocka for unit testing. Install it so that the CI can
use it.

Change-Id: I5c168e480d6f4cbfbbd175ecb035c88bfcbac00b
Signed-off-by: Felix Singer <felixsinger@posteo.net>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/67272
Reviewed-by: Angel Pons <th3fanbus@gmail.com>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-09-02 18:48:13 +00:00
Martin Roth b3f91b7941 util/docker: Update dockerfiles
- Remove deprecated "MAINTAINER" lines
- Add Sphinx tools to coreboot-jenkins-node to check documentation.
- Add mdl to check markdown
- Alphabetize packages in docs Dockerfile
- Add jinja2 version 3.0.3 to the docs Dockerfile - The latest version
breaks with the error:
"exception: cannot import name 'contextfunction' from 'jinja2'"

Signed-off-by: Martin Roth <gaumless@gmail.com>
Change-Id: Ia1de62621a6aef4ecd055a1a3afbebad34448002
Reviewed-on: https://review.coreboot.org/c/coreboot/+/64655
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Felix Singer <felixsinger@posteo.net>
2022-05-31 13:41:56 +00:00
Martin Roth 1fcf78cc8e util/docker/coreboot-jenkins-node: Alphabetize installed tools
It's easier to read and to add new packages when each package is on its
own line and they're sorted alphabetically.

Indenting them also makes it easier to see what's getting installed and
what's a command.

Signed-off-by: Martin Roth <martin@coreboot.org>
Change-Id: Ibfe297bd408ed0783fcff09c1ecb5672fe785c48
Reviewed-on: https://review.coreboot.org/c/coreboot/+/62446
Reviewed-by: Felix Singer <felixsinger@posteo.net>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-03-02 13:09:55 +00:00
Martin Roth c8a1195b77 util/docker/coreboot-jenkins-node: add linkchecker
The linkchecker tool is now being used to find broken links in our
websites.  Since it's not needed for building anything, just add it to
the jenkins-node Dockerfile instead.

Signed-off-by: Martin Roth <martin@coreboot.org>
Change-Id: Iac2246b5378e556b5cd9f2107fc5a7e51d583b5b
Reviewed-on: https://review.coreboot.org/c/coreboot/+/62445
Reviewed-by: Felix Singer <felixsinger@posteo.net>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-03-02 13:09:40 +00:00
Patrick Georgi 86e9b41ac2 docker/coreboot-jenkins-node: Add more tools for zephyr
To build a CrOS-style zephyr, we need a couple of u-boot tools, so add
them here instead of rebuilding them on every zephyr build (which is
also harder to get right because search paths are no strength of python)

Change-Id: Ib95fcb644ac87c5f35f2228fe081c922452b5213
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/50744
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Martin Roth <martinroth@google.com>
2021-02-16 23:39:09 +00:00
Patrick Georgi 2fae1c0494 docker/coreboot-jenkins-node: Add GNU parallel
Change-Id: I958e65f3c758e7e46d6b628a05009c1b4727d40a
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/50087
Reviewed-by: Martin Roth <martinroth@google.com>
Reviewed-by: Angel Pons <th3fanbus@gmail.com>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2021-01-29 22:46:54 +00:00
Patrick Georgi dd5fe14759 docker/coreboot-jenkins-node: Add zephyr-sdk toolchain
There are efforts to replace Chrome EC with Zephyr. To ensure
Chromebook specific Zephyr developments (that can eventually be
built as part of a coreboot build just like Chrome EC now, and are
built with coreboot-sdk) don't break with Zephyr's toolchain, add
the toolchain to our builders so we can do some sanity checking.

Change-Id: I645a298bc350ebe7651c08aea630bdc6b93856aa
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/49986
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Martin Roth <martinroth@google.com>
2021-01-28 12:32:32 +00:00
Patrick Georgi 0a9eea0f5b util/docker: Add sdcc to our build nodes
core-ec will need it.

Change-Id: Id7d677a6f92ce266f893372a2540d77abb613707
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/46940
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Martin Roth <martinroth@google.com>
2020-10-30 09:28:41 +00:00
Martin Roth 2aba15fce1 util/docker: Update atime mount point options for jenkins
- The ccache files don't need atime.
- Enable strict atime for the git repos.  This will help find unused
files.

Signed-off-by: Martin Roth <martin@coreboot.org>
Change-Id: I94bcc55ea5c5a74f3ad0292ca50b74874a0d920d
Reviewed-on: https://review.coreboot.org/c/coreboot/+/46804
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2020-10-27 10:09:50 +00:00
Martin Roth a266299490 util/docker: Update agent-root to node-root for jenkins
Jenkins has changed the name of the build directory, so it's not
currently building out of memory, it's writing to the SSD. This
changes the build back to tmpfs.

Signed-off-by: Martin Roth <martin@coreboot.org>
Change-Id: Iefcf53757862feb2025aa5696f9f5dbce9dd70dd
Reviewed-on: https://review.coreboot.org/c/coreboot/+/46803
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2020-10-27 10:09:06 +00:00
Stefan Reinauer 88e54e38d4 Fix up Docker paths to match jenkins nomenclature
Jenkins is calling its build nodes "agent". Reflect that in
the path names we use in configuration.

Change-Id: I88a4d3d32a565ade768e3de6428f46d355bedfb2
Signed-off-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/42819
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2020-06-26 18:56:48 +00:00
Patrick Georgi 02a1344921 docker/coreboot-jenkins-node: Add meson and ninja
Our jenkins instance is also used for flashrom, which can be built with
meson, a mode that we want to be able to test, so add that.

ninja can be used as a backend to both meson and cmake (which coreboot
will use to build cmocka for its unit tests) and may provide some
additional coverage. Plus it's tiny but fast.

Change-Id: If454164852303144eaa72c4071c03ee89e863318
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/41731
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Angel Pons <th3fanbus@gmail.com>
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2020-05-28 05:41:30 +00:00
Patrick Georgi 3404247115 util/docker: Use more stable URL
The pgeorgi namespace is my own and things could change without notice
there. To overcome this issue, encapsulate is now maintained on
review.coreboot.org/encapsulate.git and mirrored over to github, so
let's use that.

Change-Id: I12e43f61f693a6b0392b84dd56ede665a1a2129a
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/38899
Reviewed-by: Martin Roth <martinroth@google.com>
Reviewed-by: Angel Pons <th3fanbus@gmail.com>
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2020-02-17 15:40:17 +00:00
Martin Roth 26c43b7a77 util/docker: Update JRE in coreboot-jenkins-node Dockerfile
openjdk-8-jre-headless is no longer available in the debian image we're
basing the coreboot-sdk off of.  Update it to 'default-jre-headless'.

Change-Id: I60f6ecbaedccc0da61f96e0bce4122406ba4bd91
Signed-off-by: Martin Roth <martinroth@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/32254
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2019-04-17 14:48:32 +00:00
Martin Roth 30b7c31547 util/docker: Update coreboot-jenkins-node dockerfile
Because earlier versions of debian set /dev as a standard tmpfs
filesystem, that was a simple place to build.  Now, this has been
changed and /dev isn't a standard tmpfs that will grow to 50% of
memory.  It's a fixed, very small size, and can no longer even be
resized.
Because of this, create a new directory to build in and add it to
/etc/fstab.  Mount it when the container is started.

As long as we're at it, make the other build directories (ccache
and slave-root/workspace) tmpfs as well. The builders we're using now
have plenty of memory, so don't write any of the files to disk.

Update the Makefile to get rid of all references to ccache directory.

Change-Id: I21fd2c4395d7ffb9428172f035991338658cd907
Signed-off-by: Martin Roth <gaumless@gmail.com>
Reviewed-on: https://review.coreboot.org/27470
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2018-07-17 11:38:55 +00:00
Martin Roth 561f368a2f util/docker: Update dockerfiles & build method
All files:
- Previously, various things were hardcoded into the docker containers
that made it necessary to update the Dockerfile files for each new
version of the sdk.  Turn those into 'Variables" that are updated during
the build step.  Because the makefile is piping the dockerfile through
the sed command and back into the docker build command, the normal
docker "COPY" keyword doesn't work.

coreboot-jenkins-node changes:
- Run ssh-keygen -A to explicitly generate the ssh keys.  This fixes an
error:  Could not load host key: /etc/ssh/ssh_host_dsa_key

coreboot-sdk changes:
- Remove apt-get upgrade command - The Dockerfile guide recommends
not to run this.
- Change libssl-dev to libssl1.0-dev. libssl-dev's header files won't
build the Chrome-EC codebase.
- Add libisl-dev, needed to build the riscv toolchain.
- Build the toolchain using the -b option
- Add environment variables containing the version and commit that the
coreboot-sdk was built from.

Makefile:
- Update targets to use the version and commit variables

Change-Id: I2c1376fe4b791da2a62fca11bc92c4774cbef1c8
Signed-off-by: Martin Roth <gaumless@gmail.com>
Reviewed-on: https://review.coreboot.org/18001
Tested-by: build bot (Jenkins)
Reviewed-by: Philipp Deppenwiese <zaolin.daisuki@gmail.com>
2017-03-06 00:30:35 +01:00
Martin Roth 2d97cb1be5 util/docker: Update docker files for coreboot-sdk & jenkins builders
- Check out the specific toolchain version we want before building
the toolchain (This version uses 1.42).
- Add additional libraries and tools needed to build coreboot related
packages.
- Move everything required to build any of the coreboot or related
packages into the coreboot-sdk from coreboot-jenkins-node Dockerfile.
- Separate the text of the commands in the Dockerfiles.
- Use nproc to get the number of processors for building the toolchain
- Add some additional comments about why things are done the way that
they are to the README
- Update the version of coreboot-sdk that coreboot-jenkins-node uses to
1.42. (This matches the toolchain version)
- Move ccache setup from jenkins-node to coreboot-sdk.
- Update the maintainer.

Change-Id: I293285ef72e3e70259355d924d425fea98ee773d
Signed-off-by: Martin Roth <martinroth@google.com>
Reviewed-on: https://review.coreboot.org/16239
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2016-08-18 06:15:26 +02:00
Martin Roth f75d1dfad3 util/docker: Add docker files for coreboot-sdk & jenkins builders
Add the coreboot specific docker configuration files to the coreboot
repo.  These have been copied directly from Patrick's repo where they
had been being stored.

- coreboot-sdk: debian sid with the coreboot toolchain
- coreboot-jenkins-node: built on top of the coreboot-sdk, adds the
pieces required for building everything with the coreboot jenkins
builders.

Change-Id: I8628d4edb298264e814e02e124a8bfb4bc04e0c7
Signed-off-by: Martin Roth <martinroth@google.com>
Reviewed-on: https://review.coreboot.org/14830
Tested-by: build bot (Jenkins)
Reviewed-by: Alexander Couzens <lynxis@fe80.eu>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2016-08-18 06:14:40 +02:00