Commit Graph

39 Commits

Author SHA1 Message Date
Jakub Czapiga a0e36d8cba tests: Add support for tests build failures detection
This patch introduces new target: junit.xml-unit-tests, which builds and
runs unit-tests. It also creates build log containing build logs. This
feature allows for one to see build failures in Jenkins dashboard.

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: I94184379dcc2ac10f1a47f4a9d205cacbeb640fe
Reviewed-on: https://review.coreboot.org/c/coreboot/+/67372
Reviewed-by: Julius Werner <jwerner@chromium.org>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-09-21 14:06:42 +00:00
Jakub Czapiga 9ebc6c1e26 tests: Split Makefile to allow for making host-side test tools
This patch is based on similar changes [1] done in Depthcharge projects,
which aimed to provide unified way to build host-side programs for
testing internal code. New test tools might benefit from it by having
same base code as unit-tests.

[1] https://crrev.com/c/3412108

TEST=make unit-tests
TEST=COV=1 make unit-tests coverage-report

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: Iac4517ab6146fa3f2d2b7a20df54601ab2d04c3d
Reviewed-on: https://review.coreboot.org/c/coreboot/+/63637
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Arthur Heymans <arthur@aheymans.xyz>
2022-04-19 13:00:56 +00:00
Julius Werner c90d406008 tests: Disable -Wmain-return-type for clang
Unit tests fail on clang 13.0 because Cmocka's main() function is
declared with return type int, but through some of our chain-include
games it sometimes gets coreboot's main_decl.h which declares the
function as void. Compilers have traditionally always been very
laissez-faire about the signature of the main function, but it seems
that clang has been getting more strict by default recently. Explicitly
disable the related warning to shut it up.

Signed-off-by: Julius Werner <jwerner@chromium.org>
Change-Id: I867c9dac659be86e7b7cf4cc41d6fa105aa9ac41
Reviewed-on: https://review.coreboot.org/c/coreboot/+/59815
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-12-07 17:47:35 +00:00
Julius Werner 3332c2dc6e tests: Disable -Wsource-mgr for clang
Unit tests fail on clang 13.0 with a cryptic

  error: _timestamp_size changed binding to STB_WEAK [-Werror,-Wsource-mgr]

Probably something related to the weird things we do to mock memlayout
areas. Too lazy to track it down. Let's just disable this
(clang-specific, not properly documented) warning flag.

Signed-off-by: Julius Werner <jwerner@chromium.org>
Change-Id: Ie49c9eef3c74592c068c899c6717621dbcb9f609
Reviewed-on: https://review.coreboot.org/c/coreboot/+/59814
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-12-07 17:47:09 +00:00
Jakub Czapiga b88d845c56 tests: Fix objcopy --add-symbol arguments creation
Remove unnecessary escape bachslashes from:
addr="$(echo \"$$$$sym_line\" ...
The 'echo' will print output line with double quote characters, which
then will be included in the values passed to --add-symbol. This can
cause errors.

Change-Id: I6023515191d6c236bf57870159a35d518f25e9d8
Reported-by: Yu-Ping Wu <yupingso@chromium.org>
Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/59858
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Yu-Ping Wu <yupingso@google.com>
2021-12-06 12:34:09 +00:00
Raul E Rangel 940953e823 tests/Makefile: Remove ./ prefix when running tests
If ran with obj=/absolute path, then tests were failing to execute
because the recipe tried running `.//absolutepath/...run`.

Signed-off-by: Raul E Rangel <rrangel@chromium.org>
Change-Id: I9c3638b1af7531dbe8e956dcbe168250a235ead4
Reviewed-on: https://review.coreboot.org/c/coreboot/+/58806
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-11-03 08:22:12 +00:00
Jakub Czapiga c1e4c5aaa5 tests: Add lib/lzma-test test case
Files used by this test are in: tests/data/lib/lzma-test/
file.bin - files with uncompressed data
file.lzma.bin - files with LZMA-compressed data from file.bin

How to prepare compressed file:
  util/cbfs-compression-tool compress file.bin /tmp/file.lzma.bin lzma
  dd if=/tmp/file.lzma.bin of=file.lzma.bin skip=8 ibs=1

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: Id75e0b41991382d4c391b031862106de58eacdf7
Reviewed-on: https://review.coreboot.org/c/coreboot/+/57555
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-10-14 13:13:07 +00:00
Jakub Czapiga 4d9694e6bc tests: Fix JUNIT_OUTPUT=y to write to files instead of stderr
CB:57144 broke JUNIT_OUTPUT=y, and unit-tests were writing to stderr
instead of selected XML files, because test name used for XML file
creation contains test path. Build system did not create necessary
directiories, which CMocka required to create output files.
This commit fixes writing to XML files with JUNIT_OUTPUT=y, but had to
sacrifice path in test name, as it was causing a lot of problems
(because CMocka does not know, how to write multiple test groups to one
XML file, so it uses test group name [here __TEST_NAME__(test_group)] as
part of output filename).
Example:
 Test: tests/lib/rtc-test
 Output file:
 `build/tests/junit-tests_lib_rtc-test(tests).xml

Change-Id: I09891aca923bf1271cafeaa09f89b6539022709c
Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/58163
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-10-11 12:55:36 +00:00
Jakub Czapiga 29faa8a5a2 tests/stubs/console: Allow enabling printk to print to stdout
By adding TEST_PRINT=1 to <test-name>-config field or by passing it as
a parameter to make one can enable printing in printk() and vprintk().
This can be helpful when developing unit tests.
Note, that to effectively enable or disable printk() printing to stdout,
test(s) have to be recompiled.

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: Ibdec8bb128f42ba4d9cb8bbb4a8c5159a2b52ac5
Reviewed-on: https://review.coreboot.org/c/coreboot/+/57526
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Yu-Ping Wu <yupingso@google.com>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-09-13 14:00:57 +00:00
Jakub Czapiga 7c6081e02b tests: Improve test output readability
When running multiple tests, e.g. by using unit-tests target, it is hard
to differentiate, which output comes from which file and/or
configuration. This patch makes the output easier to analyze and
understand by using new wrapper macro cb_run_group_tests(). This macro
uses __TEST_NAME__ value (containing test path and Makefile test name)
as a group name when calling cmocka group runner.

Example:
 Test path: tests/lib/
 Makefile test name: cbmem_stage_cache-test
 Test group array name: tests
 Result: tests/lib/cbmem_stage_cache-test(tests)

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: I4fd936d00d77cbe2637b857ba03b4a208428ea0d
Reviewed-on: https://review.coreboot.org/c/coreboot/+/57144
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-09-01 19:38:09 +00:00
Julius Werner ab91407735 tests: Fix function mocking for clang
clang seems to like to do some aggressive optimizations that break our
approach of mocking functions for test by using objcopy to turn them
weak after the fact on individual compiled object files. For example, in
CB:56601 the function cbfs_get_boot_device() is mocked this way. When
compiling the cbfs_boot_lookup() function in src/lib/cbfs.c with clang,
it will generate a normal callq instruction to a relocation for
cbfs_boot_lookup(), which can then later be pointed to the mocked
version of that function. However, it will also somehow infer that the
version of cbfs_boot_lookup() in that file can only ever return a
pointer to the static local `ro` variable (because CONFIG_VBOOT is
disabled in the environment for that particular test), and instead
generate instructions that directly load the address of a relocation for
that variable into %rdi for the following call to cbfs_lookup(), rather
than using the real function return value. (Why it would do that is
anyone's guess because this seems unlikely to be faster than just moving
the function return value from %rax into %rdi like a normal compiler.)

Long story short, this optimization breaks our tests because
cbfs_lookup() will be called with the wrong pointer. clang doesn't
provide many options to disable individual optimizations, so the only
solution seems to be to make clang aware that the function is weak
during the compilation stage already, so it can be aware that it may get
replaced. This patch implements that by marking the mocked functions
weak via #pragma weak lines in the per-test autogenerated config header.

Signed-off-by: Julius Werner <jwerner@chromium.org>
Change-Id: I1f9011f444248544de7a71bbefc54edc006ae0cd
Reviewed-on: https://review.coreboot.org/c/coreboot/+/57009
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Menzel <paulepanter@mailbox.org>
Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-08-25 20:54:16 +00:00
Jakub Czapiga d0bd012b5e tests/Makefile.inc: Add missing include paths to TEST_CFLAGS
Add missing paths to common tests cflags and remove these paths from
individual tests configuration.

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: I13cb336195bfb06b861d7f403822f06bec8a40aa
Reviewed-on: https://review.coreboot.org/c/coreboot/+/56933
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-13 18:05:55 +00:00
Jakub Czapiga 9bf716ede0 tests/Makefile.inc: Add copy-test macro
copy-test macro copies attributes from one test to another. It can be
used to create multiple tests with the same subset of attributes values.

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: I9c9b6b12830c7060ffe5dcf35c9486655cbd08e0
Reviewed-on: https://review.coreboot.org/c/coreboot/+/56932
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-13 18:05:41 +00:00
Jakub Czapiga eaf5ff3607 tests/Makefile.inc: Change assignments to immediate and fix style
Change assignments to be immediate instead of lazy. Add spaces between
variable name and assignment operators to increase readability.

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d
Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-13 18:05:21 +00:00
Jakub Czapiga a7a49e64c8 tests/Makefile.inc: Add function wrapping mechanism
This patch extends mocks functionality to allowing wrapping of mocked
functions. Original function name will be prefixed with `__real_`.
Example:
- Mocked function: cbfs_lookup()
- New function name: __real_cbfs_lookup()
- Mock name: cbfs_lookup()

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: I7cd0d66a17029955cbf75c8b155a7ebb7f5513aa
Reviewed-on: https://review.coreboot.org/c/coreboot/+/56719
Reviewed-by: Julius Werner <jwerner@chromium.org>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2021-08-09 18:01:54 +00:00
Yu-Ping Wu 0f88f6ff92 tests/Makefile.inc: Correct dependency file suffix
The dependency file, generated by

 $(HOSTCC) ... -MMD -MT $@ -c $< -o $@.orig

is determined by replacing the suffix of "$@.orig" with ".d", resulting
in "$@.d" (which is *.o.d). The file name was accidentally changed in
CB:55360. Now explicitly specify the path by the "-MF" option.

BUG=none
TEST=make unit-tests; find build/tests/ -name "*.d"
BRANCH=none

Change-Id: I01f77ebaaae78dd9e69394a49e524f1013857195
Signed-off-by: Yu-Ping Wu <yupingso@chromium.org>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/56444
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-07-21 16:22:21 +00:00
Jakub Czapiga 21863e33c8 tests/Makefile.inc: Add missing KCONFIG_SPITCONFIG trailing slash
New version of kconfig requires trailing slash at the end of
KCONFIG_SPLITCONFIG to indicate that it is a directory path.

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: I1a11eca21c4aa5a6260006c4ba2cb419eca8a802
Reviewed-on: https://review.coreboot.org/c/coreboot/+/56337
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2021-07-20 16:07:15 +00:00
Patrick Georgi 53ea1d44f0 util/kconfig: Uprev to Linux 5.13's kconfig
This was originally several commits that had to be squashed into one
because the intermediate states weren't able to build coreboot:

 - one to remove everything that wasn't our own code, leaving only
   regex.[ch], toada.c, description.md and Makefile.inc.
 - one to copy in Linux 5.13's scripts/kconfig and adapt Makefile.inc
   to make the original Makefile work again.
 - adapt abuild to use olddefconfig, simplifying matters.
 - apply patches in util/kconfig/patches.
 - Some more adaptations to the libpayload build system.

The patches are now in util/kconfig/patches/, reverse applying them
should lead to a util/kconfig/ tree that contains exactly the Linux
version + our own 5 files.

Change-Id: Ia0e8fe4e9022b278f34ab113a433ef4d45e5c355
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/37152
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Raul Rangel <rrangel@chromium.org>
2021-07-13 20:28:14 +00:00
Patrick Georgi ce55ca2fca tests: Rework mocking facility
Using the linker's --wrap feature has the downside that it only covers
references across object files: If foo.c defines a() and b(), with b
calling a, --wrap=a does nothing to that call.

Instead, use objcopy to mark a weak and global so it can be overridden
by another implementation, but only for files originating in src/.

That way mocks - implemented in tests/ - become the source of truth.

TEST=Had such an issue with get_log_level() in a follow-up commit, and
the mock now takes over. Also, all existing unit tests still pass.

Change-Id: I99c6d6e44ecfc73366bf464d9c51c7da3f8db388
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/55360
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-06-10 19:26:46 +00:00
Jakub Czapiga 203698a4cb tests/Makefile.inc: Move generated headers to corresponding build dir
Test-local config override headers were generated to paths missing
/tests/ infix, thus creating divergent tree in build output directory.
This patch fixes it moving generated config headers to the test-local
build directory.

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: Ic5f3ba287ba3e9f5897cbaac64e88c2809f52d73
Reviewed-on: https://review.coreboot.org/c/coreboot/+/54917
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-05-30 20:28:30 +00:00
Paul Fagerburg de6cbac3c4 tests: improve code coverage support
Fix the exclusion path for lcov; it should exclude the directory
with source code, not object files.

Use the COV environment variable to
* control whether we build for coverage or not
* select the output directory

Add a separate target for generating the report, so we can get a
report for all of the tests together or just a single test.

Add documentation.

Signed-off-by: Paul Fagerburg <pfagerburg@google.com>
Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a
Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-19 19:56:02 +00:00
Jakub Czapiga 5c3b05ecf4 tests: Enable config override for tests
Some tests require to change kconfig symbols values to cover the code.
This patch enables one to set these vaues using <test-name>-config
variable.

Example for integer values.
timestamp-test-config += CONFIG_HAVE_MONOTONIC_TIMER=1

Example for string values. Notice escaped quotes.
spd_cache-test-config += CONFIG_SPD_CACHE_FMAP_NAME=\"SPD_CACHE_FMAP_NAME\"

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: I1aeb78362c2609fbefbfd91c0f58ec19ed258ee1
Reviewed-on: https://review.coreboot.org/c/coreboot/+/52937
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-05-13 00:15:54 +00:00
Paul Fagerburg 045fbf138a tests: enable code coverage for unit tests
Add a new `coverage-unit-tests` make target that builds the unit
tests for code coverage, runs the tests, and generates a coverage
report.

Signed-off-by: Paul Fagerburg <pfagerburg@google.com>
Change-Id: I6ea780ee9e246c0bb8c35b8e0de4252431dabbff
Reviewed-on: https://review.coreboot.org/c/coreboot/+/52444
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-05 11:45:19 +00:00
Raul E Rangel 66e35fb341 Makefile,tests: Move cmocka checkout into top level Makefile
cmocka is currently ignoring the UPDATED_SUBMODULES flag. Move the
cmocka checkout with the other submodule checkouts.

BUG=none
TEST=Make sure cmocka is not checked out if UPDATED_SUBMODULES=1

Signed-off-by: Raul E Rangel <rrangel@chromium.org>
Change-Id: I2a1db809368a77d2c0f9c9a796d62555ec476dc7
Reviewed-on: https://review.coreboot.org/c/coreboot/+/52578
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-04-30 12:43:44 +00:00
Jakub Czapiga a4819b3c31 Makefile: Add unit-tests help and targets list
Add unit-tests targets to help output. Add list-unit-tests target
that lists all available unit-tests.

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: I464a76cbea1f4afbc3fc772960787952e61b95b9
Reviewed-on: https://review.coreboot.org/c/coreboot/+/52293
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-04-15 07:38:48 +00:00
Jakub Czapiga 084ad93663 include/rules.h: Add ENV_TEST definition
Some functions/macros like assert() require redefinition for testing
purposes. ENV_TEST is introduced to make it possible without using
bypass hacks.
This patch also adds a global __TEST__ define to TEST_CFLAGS for
all test targets in order to enable ENV_TEST.

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: Ib8f2932902a73a7dbe181adc82cc18437abb48e8
Reviewed-on: https://review.coreboot.org/c/coreboot/+/51803
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-04-06 06:58:05 +00:00
Jakub Czapiga 1add483819 tests/Makefile.inc: Enable support for multiple test groups
Until now output of all test groups run in single unit test were
saved in the same file which caused Jenkins to fail because
of existence of multiple root XML elements.
Now each test group is saved to its own file containing its name
at the end of the filename.

Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: I21ba512073bc8d8693daad8a9b86d5b076bea03f
Reviewed-on: https://review.coreboot.org/c/coreboot/+/51281
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-03-10 20:23:19 +00:00
Jakub Czapiga 168b38bbb3 tests: Add acpi/acpigen-test test case
Signed-off-by: Jakub Czapiga <jacz@semihalf.com>
Change-Id: Icc128212c1f72beb50caca671b4bada3507d3a1f
Reviewed-on: https://review.coreboot.org/c/coreboot/+/50520
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2021-02-23 17:10:10 +00:00
Julius Werner 84446e6e54 rmodtool: Make memlayout symbols absolute and do not relocate them
Memlayout is a mechanism to define memory areas outside the normal
program segment constructed by the linker. Therefore, it generally
doesn't make sense to relocate memlayout symbols when the program is
relocated. They tend to refer to things that are always in one specific
spot, independent of where the program is loaded.

This hasn't really hurt us in the past because the use case we have for
rmodules (ramstage on x86) just happens to not really need to refer to
any memlayout-defined areas at the moment. But that use case may come up
in the future so it's still worth fixing.

This patch declares all memlayout-defined symbols as ABSOLUTE() in the
linker, which is then reflected in the symbol table of the generated
ELF. We can then use that distinction to have rmodtool skip them when
generating the relocation table for an rmodule. (Also rearrange rmodtool
a little to make the primary string table more easily accessible to the
rest of the code, so we can refer to symbol names in debug output.)

A similar problem can come up with userspace unit tests, but we cannot
modify the userspace relocation toolchain (and for unfortunate
historical reasons, it tries to relocate even absolute symbols). We'll
just disable PIC and make those binaries fully static to avoid that
issue.

Signed-off-by: Julius Werner <jwerner@chromium.org>
Change-Id: Ic51d9add3dc463495282b365c1b6d4a9bf11dbf2
Reviewed-on: https://review.coreboot.org/c/coreboot/+/50629
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2021-02-18 02:32:06 +00:00
Julius Werner e54d784d02 tests: Build tests with -Wno-inline-asm
Clang doesn't seem to get along with some of the symbol magic we use for
memlayout and throws -Winline-asm warnings. Since we want to be
compatible with as many host compilers as possible (within reason),
let's disable that warning.

Signed-off-by: Julius Werner <jwerner@chromium.org>
Change-Id: If1d88ed0bb2d10acfadcf8dec74fa3d227e0f790
Reviewed-on: https://review.coreboot.org/c/coreboot/+/50825
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jan Dabros <jsd@semihalf.com>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-02-17 23:10:15 +00:00
Julius Werner 1e14de8bda tests: Add some basic warnings and fix resulting issues
The current test framework builds the test code without any warnings at
all, which isn't great -- we have already slipped in some cases of
non-void functions not returning a defined value, for example. It would
likely be overkill to try to use all the same warnings we use for normal
coreboot code (e.g. some stuff like -Wmissing-prototypes makes cmocka's
__wrap_xxx() mock functions unnecessarily cumbersome to work with, and
other things like -Wvla may be appropriate for firmware but is probably
too aggressive for some simple test code). Therefore, let's just add
some of the stuff that points out the most obvious errors.

Signed-off-by: Julius Werner <jwerner@chromium.org>
Change-Id: I4d9801f52a8551f55f419f4141dc21ccb835d676
Reviewed-on: https://review.coreboot.org/c/coreboot/+/42259
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jan Dabros <jsd@semihalf.com>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2020-06-12 19:40:19 +00:00
Patrick Georgi 1b35ec97cc tests: Always run all unit tests
So far, the semantics have been that run-unit-tests stopped at the first
test suite that failed. This hides useful signal in later tests, so
always run all tests and collect the result.

Change-Id: I407715f85513c2c95a1cf89cfb427317dff9fbab
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/41773
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jan Dabros <jsd@semihalf.com>
Reviewed-by: Martin Roth <martinroth@google.com>
Reviewed-by: Angel Pons <th3fanbus@gmail.com>
2020-05-28 09:48:13 +00:00
Patrick Georgi a0a198f779 tests: Allow emitting junit output for unit tests
Change-Id: Iab0c4250b1baa77d4eab7538ec1fd3310f9e63e4
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/41771
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Martin Roth <martinroth@google.com>
Reviewed-by: Angel Pons <th3fanbus@gmail.com>
Reviewed-by: Jan Dabros <jsd@semihalf.com>
2020-05-28 09:47:44 +00:00
Patrick Georgi a3402973ab tests: Move cmocka binary into $(obj)
Put it in $(objutil) so that it's shared between board builds with
abuild even if that doesn't matter right now.

Change-Id: I5670d9b661891262ad936980f63fa93b07c27e95
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/41770
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jan Dabros <jsd@semihalf.com>
Reviewed-by: Martin Roth <martinroth@google.com>
Reviewed-by: Angel Pons <th3fanbus@gmail.com>
2020-05-28 09:47:36 +00:00
Jan Dabros 7f00dba33b tests: Build Cmocka from source
Relying on Cmocka packages, which are provided with different OS
distributions, may introduce some problems with setup environments
across developers (e.g. library version mismatch). Instead, let's build
Cmocka from source code, which is now added to git submodules as
3rdparty/cmocka.

Please note, that cmake tool is required for building Cmocka (thus also
coreboot unit tests).

Signed-off-by: Jan Dabros <jsd@semihalf.com>
Change-Id: Ia947c5c60d5c58b76acebe4b614dd427ef995950
Reviewed-on: https://review.coreboot.org/c/coreboot/+/41653
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2020-05-26 16:21:06 +00:00
Julius Werner 2540037406 tests: Add region-test for rdev API
This patch adds a basic test for the common region and region_device
APIs, sanity checking the basic functions and things like
overflow-handling. There is certainly more that could be added here, but
it's a start.

Signed-off-by: Julius Werner <jwerner@chromium.org>
Change-Id: I4932402f54768557e5b22b16e66220bd90ddebfd
Reviewed-on: https://review.coreboot.org/c/coreboot/+/41046
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-by: Jan Dabros
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2020-05-19 03:22:04 +00:00
Julius Werner 21744811bb tests: Add <tests/test.h> wrapper header and fix --gc-sections
<cmocka.h> requires a few standard headers to be explicitly included
before itself or it will throw compilation errors. Having to always
include these headers in the right order in every test is cumbersome.
Instead, this patch encapsulates the problem in a new <tests/test.h>
header that all tests should include (instead of <cmocka.h> directly).

Also fix --gc-sections in the test framework which needs to be passed
for linking, not for compiling.

Signed-off-by: Julius Werner <jwerner@chromium.org>
Change-Id: I4284d74c8673708e21a5266eb42f7b9ae19a1b12
Reviewed-on: https://review.coreboot.org/c/coreboot/+/41045
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Jan Dabros
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2020-05-19 03:21:44 +00:00
Patrick Georgi 6b5bc77c9b treewide: Remove "this file is part of" lines
Stefan thinks they don't add value.

Command used:
sed -i -e '/file is part of /d' $(git grep "file is part of " |egrep ":( */\*.*\*/\$|#|;#|-- | *\* )" | cut -d: -f1 |grep -v crossgcc |grep -v gcov | grep -v /elf.h |grep -v nvramtool)

The exceptions are for:
 - crossgcc (patch file)
 - gcov (imported from gcc)
 - elf.h (imported from GNU's libc)
 - nvramtool (more complicated header)

The removed lines are:
-       fmt.Fprintln(f, "/* This file is part of the coreboot project. */")
-# This file is part of a set of unofficial pre-commit hooks available
-/* This file is part of coreboot */
-# This file is part of msrtool.
-/* This file is part of msrtool. */
- * This file is part of ncurses, designed to be appended after curses.h.in
-/* This file is part of pgtblgen. */
- * This file is part of the coreboot project.
- /* This file is part of the coreboot project. */
-#  This file is part of the coreboot project.
-# This file is part of the coreboot project.
-## This file is part of the coreboot project.
--- This file is part of the coreboot project.
-/* This file is part of the coreboot project */
-/* This file is part of the coreboot project. */
-;## This file is part of the coreboot project.
-# This file is part of the coreboot project. It originated in the
- * This file is part of the coreinfo project.
-## This file is part of the coreinfo project.
- * This file is part of the depthcharge project.
-/* This file is part of the depthcharge project. */
-/* This file is part of the ectool project. */
- * This file is part of the GNU C Library.
- * This file is part of the libpayload project.
-## This file is part of the libpayload project.
-/* This file is part of the Linux kernel. */
-## This file is part of the superiotool project.
-/* This file is part of the superiotool project */
-/* This file is part of uio_usbdebug */

Change-Id: I82d872b3b337388c93d5f5bf704e9ee9e53ab3a9
Signed-off-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/41194
Reviewed-by: HAOUAS Elyes <ehaouas@noos.fr>
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2020-05-11 17:11:40 +00:00
Jan Dabros ef1c968374 tests: Add build subsystem for unit testing coreboot
Add a subsystem which will be used for writing, building and running
unit tests for different coreboot's modules. This work is built using
Cmocka unit testing framework. Description of what unit testing means
(for the author) and how unit testing framework evaluation was performed
may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md

Makefiles structure is very similar to this used for building coreboot
images. Every directory has its own Makefile.inc were tests' names,
sources, subdirs and multiple other test-related attributes are defined
in form of variables.

Signed-off-by: Jan Dabros <jsd@semihalf.com>
Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1
Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-05-01 06:32:47 +00:00