GNUBoot
/
coreboot-kgpe-d16
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
coreboot-kgpe-d16/tests/Makefile.inc

301 lines
10 KiB
PHP
Raw Normal View History

tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
# SPDX-License-Identifier: GPL-2.0-only
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
testsrc := $(top)/tests
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
# Place the build output in one of two places depending on COV, so that code
# built with code coverage never mixes with code built without code coverage.
ifeq ($(COV),1)
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
testobj := $(obj)/coverage
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
else
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
testobj := $(obj)/tests
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
endif
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
cmockasrc := 3rdparty/cmocka
cmockaobj := $(objutil)/cmocka
coverage_dir := coverage_reports
tests: Build Cmocka from source Relying on Cmocka packages, which are provided with different OS distributions, may introduce some problems with setup environments across developers (e.g. library version mismatch). Instead, let's build Cmocka from source code, which is now added to git submodules as 3rdparty/cmocka. Please note, that cmake tool is required for building Cmocka (thus also coreboot unit tests). Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: Ia947c5c60d5c58b76acebe4b614dd427ef995950 Reviewed-on: https://review.coreboot.org/c/coreboot/+/41653 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2020-05-22 09:57:17 +02:00
CMOCKA_LIB := $(cmockaobj)/src/libcmocka.so
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
CMAKE := cmake
OBJCOPY ?= objcopy
OBJDUMP ?= objdump
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
TEST_DEFAULT_CONFIG := $(top)/configs/config.emulation_qemu_x86_i440fx
TEST_DOTCONFIG := $(testobj)/.config
tests: Enable config override for tests Some tests require to change kconfig symbols values to cover the code. This patch enables one to set these vaues using <test-name>-config variable. Example for integer values. timestamp-test-config += CONFIG_HAVE_MONOTONIC_TIMER=1 Example for string values. Notice escaped quotes. spd_cache-test-config += CONFIG_SPD_CACHE_FMAP_NAME=\"SPD_CACHE_FMAP_NAME\" Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I1aeb78362c2609fbefbfd91c0f58ec19ed258ee1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/52937 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-04-28 16:50:51 +02:00
TEST_KCONFIG_AUTOHEADER := $(testobj)/config.src.h
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
TEST_KCONFIG_AUTOCONFIG := $(testobj)/auto.conf
TEST_KCONFIG_DEPENDENCIES := $(testobj)/auto.conf.cmd
tests/Makefile.inc: Add missing KCONFIG_SPITCONFIG trailing slash New version of kconfig requires trailing slash at the end of KCONFIG_SPLITCONFIG to indicate that it is a directory path. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I1a11eca21c4aa5a6260006c4ba2cb419eca8a802 Reviewed-on: https://review.coreboot.org/c/coreboot/+/56337 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2021-07-15 12:54:27 +02:00
TEST_KCONFIG_SPLITCONFIG := $(testobj)/config/
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
TEST_KCONFIG_TRISTATE := $(testobj)/tristate.conf
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
TEST_CFLAGS := -include $(src)/include/kconfig.h \
tests: Add <tests/test.h> wrapper header and fix --gc-sections <cmocka.h> requires a few standard headers to be explicitly included before itself or it will throw compilation errors. Having to always include these headers in the right order in every test is cumbersome. Instead, this patch encapsulates the problem in a new <tests/test.h> header that all tests should include (instead of <cmocka.h> directly). Also fix --gc-sections in the test framework which needs to be passed for linking, not for compiling. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: I4284d74c8673708e21a5266eb42f7b9ae19a1b12 Reviewed-on: https://review.coreboot.org/c/coreboot/+/41045 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jan Dabros Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2020-05-05 02:44:04 +02:00
-include $(src)/commonlib/bsd/include/commonlib/bsd/compiler.h \
-include $(src)/include/rules.h
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
# Include generic test mock headers, before original ones
tests: Add <tests/test.h> wrapper header and fix --gc-sections <cmocka.h> requires a few standard headers to be explicitly included before itself or it will throw compilation errors. Having to always include these headers in the right order in every test is cumbersome. Instead, this patch encapsulates the problem in a new <tests/test.h> header that all tests should include (instead of <cmocka.h> directly). Also fix --gc-sections in the test framework which needs to be passed for linking, not for compiling. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: I4284d74c8673708e21a5266eb42f7b9ae19a1b12 Reviewed-on: https://review.coreboot.org/c/coreboot/+/41045 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jan Dabros Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2020-05-05 02:44:04 +02:00
TEST_CFLAGS += -I$(testsrc)/include/mocks -I$(testsrc)/include
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
tests/Makefile.inc: Add missing include paths to TEST_CFLAGS Add missing paths to common tests cflags and remove these paths from individual tests configuration. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I13cb336195bfb06b861d7f403822f06bec8a40aa Reviewed-on: https://review.coreboot.org/c/coreboot/+/56933 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 15:51:37 +02:00
TEST_CFLAGS += -I$(src) -I$(src)/include -I$(src)/commonlib/include \
-I$(src)/commonlib/bsd/include -I$(src)/arch/x86/include \
-I$(top)/3rdparty/vboot/firmware/include
tests: Add some basic warnings and fix resulting issues The current test framework builds the test code without any warnings at all, which isn't great -- we have already slipped in some cases of non-void functions not returning a defined value, for example. It would likely be overkill to try to use all the same warnings we use for normal coreboot code (e.g. some stuff like -Wmissing-prototypes makes cmocka's __wrap_xxx() mock functions unnecessarily cumbersome to work with, and other things like -Wvla may be appropriate for firmware but is probably too aggressive for some simple test code). Therefore, let's just add some of the stuff that points out the most obvious errors. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: I4d9801f52a8551f55f419f4141dc21ccb835d676 Reviewed-on: https://review.coreboot.org/c/coreboot/+/42259 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jan Dabros <jsd@semihalf.com> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2020-06-11 00:35:08 +02:00
# Note: This is intentionally just a subset of the warnings in the toplevel
# Makefile.inc. We don't need to be as strict with test code, and things like
# -Wmissing-prototypes just make working with the test framework cumbersome.
# Only put conservative warnings here that really detect code that's obviously
# unintentional.
tests: Build tests with -Wno-inline-asm Clang doesn't seem to get along with some of the symbol magic we use for memlayout and throws -Winline-asm warnings. Since we want to be compatible with as many host compilers as possible (within reason), let's disable that warning. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: If1d88ed0bb2d10acfadcf8dec74fa3d227e0f790 Reviewed-on: https://review.coreboot.org/c/coreboot/+/50825 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jan Dabros <jsd@semihalf.com> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-02-17 04:32:24 +01:00
TEST_CFLAGS += -Wall -Werror -Wundef -Wstrict-prototypes -Wno-inline-asm
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
# Path for Kconfig autoheader
TEST_CFLAGS += -I$(dir $(TEST_KCONFIG_AUTOHEADER))
tests: Add region-test for rdev API This patch adds a basic test for the common region and region_device APIs, sanity checking the basic functions and things like overflow-handling. There is certainly more that could be added here, but it's a start. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: I4932402f54768557e5b22b16e66220bd90ddebfd Reviewed-on: https://review.coreboot.org/c/coreboot/+/41046 Reviewed-by: Aaron Durbin <adurbin@chromium.org> Reviewed-by: Jan Dabros Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2020-05-05 02:46:31 +02:00
TEST_CFLAGS += -std=gnu11 -Os -ffunction-sections -fdata-sections -fno-builtin
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
include/rules.h: Add ENV_TEST definition Some functions/macros like assert() require redefinition for testing purposes. ENV_TEST is introduced to make it possible without using bypass hacks. This patch also adds a global __TEST__ define to TEST_CFLAGS for all test targets in order to enable ENV_TEST. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Ib8f2932902a73a7dbe181adc82cc18437abb48e8 Reviewed-on: https://review.coreboot.org/c/coreboot/+/51803 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-03-25 13:10:31 +01:00
TEST_CFLAGS += -D__TEST__
tests: Build Cmocka from source Relying on Cmocka packages, which are provided with different OS distributions, may introduce some problems with setup environments across developers (e.g. library version mismatch). Instead, let's build Cmocka from source code, which is now added to git submodules as 3rdparty/cmocka. Please note, that cmake tool is required for building Cmocka (thus also coreboot unit tests). Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: Ia947c5c60d5c58b76acebe4b614dd427ef995950 Reviewed-on: https://review.coreboot.org/c/coreboot/+/41653 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2020-05-22 09:57:17 +02:00
TEST_CFLAGS += -I$(cmockasrc)/include
tests/stubs/console: Allow enabling printk to print to stdout By adding TEST_PRINT=1 to <test-name>-config field or by passing it as a parameter to make one can enable printing in printk() and vprintk(). This can be helpful when developing unit tests. Note, that to effectively enable or disable printk() printing to stdout, test(s) have to be recompiled. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Ibdec8bb128f42ba4d9cb8bbb4a8c5159a2b52ac5 Reviewed-on: https://review.coreboot.org/c/coreboot/+/57526 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Yu-Ping Wu <yupingso@google.com> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-09-09 12:11:39 +02:00
ifneq ($(filter-out 0,$(TEST_PRINT)),)
TEST_CFLAGS += -DTEST_PRINT=1
endif
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
# Link against Cmocka
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
TEST_LDFLAGS := -L$(cmockaobj)/src -lcmocka -Wl,-rpath=$(cmockaobj)/src
tests: Build Cmocka from source Relying on Cmocka packages, which are provided with different OS distributions, may introduce some problems with setup environments across developers (e.g. library version mismatch). Instead, let's build Cmocka from source code, which is now added to git submodules as 3rdparty/cmocka. Please note, that cmake tool is required for building Cmocka (thus also coreboot unit tests). Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: Ia947c5c60d5c58b76acebe4b614dd427ef995950 Reviewed-on: https://review.coreboot.org/c/coreboot/+/41653 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2020-05-22 09:57:17 +02:00
TEST_LDFLAGS += -Wl,--gc-sections
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
rmodtool: Make memlayout symbols absolute and do not relocate them Memlayout is a mechanism to define memory areas outside the normal program segment constructed by the linker. Therefore, it generally doesn't make sense to relocate memlayout symbols when the program is relocated. They tend to refer to things that are always in one specific spot, independent of where the program is loaded. This hasn't really hurt us in the past because the use case we have for rmodules (ramstage on x86) just happens to not really need to refer to any memlayout-defined areas at the moment. But that use case may come up in the future so it's still worth fixing. This patch declares all memlayout-defined symbols as ABSOLUTE() in the linker, which is then reflected in the symbol table of the generated ELF. We can then use that distinction to have rmodtool skip them when generating the relocation table for an rmodule. (Also rearrange rmodtool a little to make the primary string table more easily accessible to the rest of the code, so we can refer to symbol names in debug output.) A similar problem can come up with userspace unit tests, but we cannot modify the userspace relocation toolchain (and for unfortunate historical reasons, it tries to relocate even absolute symbols). We'll just disable PIC and make those binaries fully static to avoid that issue. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: Ic51d9add3dc463495282b365c1b6d4a9bf11dbf2 Reviewed-on: https://review.coreboot.org/c/coreboot/+/50629 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2021-02-13 02:37:27 +01:00
# Some memlayout symbols don't work with userspace relocation -- disable it.
TEST_CFLAGS += -fno-pie -fno-pic
TEST_LDFLAGS += -no-pie
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
# Enable code coverage if COV=1
ifeq ($(COV),1)
TEST_CFLAGS += --coverage
TEST_LDFLAGS += --coverage
endif
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
# Extra attributes for unit tests, declared per test
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
attributes := srcs cflags config mocks stage
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
stages := decompressor bootblock romstage smm verstage
stages += ramstage rmodule postcar libagesa
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
alltests :=
subdirs := tests/arch tests/acpi tests/commonlib tests/console tests/cpu
subdirs += tests/device tests/drivers tests/ec tests/lib tests/mainboard
subdirs += tests/northbridge tests/security tests/soc tests/southbridge
subdirs += tests/superio tests/vendorcode
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
define tests-handler
alltests += $(1)$(2)
$(foreach attribute,$(attributes),
$(eval $(1)$(2)-$(attribute) += $($(2)-$(attribute))))
$(foreach attribute,$(attributes),
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval $(2)-$(attribute) := ))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
# Sanity check for stage attribute value
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval $(1)$(2)-stage := $(if $($(1)$(2)-stage),$($(1)$(2)-stage),ramstage))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(if $(findstring $($(1)$(2)-stage), $(stages)),,
$(error Wrong $(1)$(2)-stage value $($(1)$(2)-stage). \
Check your $(dir $(1)$(2))Makefile.inc))
endef
tests/Makefile.inc: Add copy-test macro copy-test macro copies attributes from one test to another. It can be used to create multiple tests with the same subset of attributes values. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I9c9b6b12830c7060ffe5dcf35c9486655cbd08e0 Reviewed-on: https://review.coreboot.org/c/coreboot/+/56932 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 15:00:43 +02:00
# Copy attributes of one test to another.
# $1 - input test name
# $2 - output test name
copy-test = $(foreach attr,$(attributes), \
$(eval $(strip $(2))-$(attr) := $($(strip $(1))-$(attr))))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(call add-special-class, tests)
$(call evaluate_subdirs)
# Create actual targets for unit test binaries
# $1 - test name
define TEST_CC_template
tests: Enable config override for tests Some tests require to change kconfig symbols values to cover the code. This patch enables one to set these vaues using <test-name>-config variable. Example for integer values. timestamp-test-config += CONFIG_HAVE_MONOTONIC_TIMER=1 Example for string values. Notice escaped quotes. spd_cache-test-config += CONFIG_SPD_CACHE_FMAP_NAME=\"SPD_CACHE_FMAP_NAME\" Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I1aeb78362c2609fbefbfd91c0f58ec19ed258ee1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/52937 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-04-28 16:50:51 +02:00
tests: Fix function mocking for clang clang seems to like to do some aggressive optimizations that break our approach of mocking functions for test by using objcopy to turn them weak after the fact on individual compiled object files. For example, in CB:56601 the function cbfs_get_boot_device() is mocked this way. When compiling the cbfs_boot_lookup() function in src/lib/cbfs.c with clang, it will generate a normal callq instruction to a relocation for cbfs_boot_lookup(), which can then later be pointed to the mocked version of that function. However, it will also somehow infer that the version of cbfs_boot_lookup() in that file can only ever return a pointer to the static local `ro` variable (because CONFIG_VBOOT is disabled in the environment for that particular test), and instead generate instructions that directly load the address of a relocation for that variable into %rdi for the following call to cbfs_lookup(), rather than using the real function return value. (Why it would do that is anyone's guess because this seems unlikely to be faster than just moving the function return value from %rax into %rdi like a normal compiler.) Long story short, this optimization breaks our tests because cbfs_lookup() will be called with the wrong pointer. clang doesn't provide many options to disable individual optimizations, so the only solution seems to be to make clang aware that the function is weak during the compilation stage already, so it can be aware that it may get replaced. This patch implements that by marking the mocked functions weak via #pragma weak lines in the per-test autogenerated config header. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: I1f9011f444248544de7a71bbefc54edc006ae0cd Reviewed-on: https://review.coreboot.org/c/coreboot/+/57009 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Menzel <paulepanter@mailbox.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-08-18 04:50:22 +02:00
# Generate custom config.h redefining given config symbols, and declaring mocked
# functions weak. It is important that the compiler already sees that they are
# weak (and they aren't just turned weak at a later stage) to prevent certain
# optimizations that would break if the function gets replaced. (For clang this
# file needs to be marked `system_header` to prevent it from warning about
# #pragma weak entries without a matching function declaration, since there's no
# -Wno-xxx command line option for that.)
tests/Makefile.inc: Move generated headers to corresponding build dir Test-local config override headers were generated to paths missing /tests/ infix, thus creating divergent tree in build output directory. This patch fixes it moving generated config headers to the test-local build directory. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Ic5f3ba287ba3e9f5897cbaac64e88c2809f52d73 Reviewed-on: https://review.coreboot.org/c/coreboot/+/54917 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-05-25 15:56:36 +02:00
$(1)-config-file := $(testobj)/$(1)/config.h
tests: Enable config override for tests Some tests require to change kconfig symbols values to cover the code. This patch enables one to set these vaues using <test-name>-config variable. Example for integer values. timestamp-test-config += CONFIG_HAVE_MONOTONIC_TIMER=1 Example for string values. Notice escaped quotes. spd_cache-test-config += CONFIG_SPD_CACHE_FMAP_NAME=\"SPD_CACHE_FMAP_NAME\" Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I1aeb78362c2609fbefbfd91c0f58ec19ed258ee1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/52937 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-04-28 16:50:51 +02:00
$$($(1)-config-file): $(TEST_KCONFIG_AUTOHEADER)
mkdir -p $$(dir $$@)
printf '// File generated by tests/Makefile.inc\n// Do not change\n' > $$@
printf '#include <%s>\n\n' "$(notdir $(TEST_KCONFIG_AUTOHEADER))" >> $$@
for kv in $$($(1)-config); do \
key="`echo $$$$kv | cut -d '=' -f -1`"; \
value="`echo $$$$kv | cut -d '=' -f 2-`"; \
printf '#undef %s\n' "$$$$key" >> $$@; \
printf '#define %s %s\n\n' "$$$$key" "$$$$value" >> $$@; \
done
tests: Fix function mocking for clang clang seems to like to do some aggressive optimizations that break our approach of mocking functions for test by using objcopy to turn them weak after the fact on individual compiled object files. For example, in CB:56601 the function cbfs_get_boot_device() is mocked this way. When compiling the cbfs_boot_lookup() function in src/lib/cbfs.c with clang, it will generate a normal callq instruction to a relocation for cbfs_boot_lookup(), which can then later be pointed to the mocked version of that function. However, it will also somehow infer that the version of cbfs_boot_lookup() in that file can only ever return a pointer to the static local `ro` variable (because CONFIG_VBOOT is disabled in the environment for that particular test), and instead generate instructions that directly load the address of a relocation for that variable into %rdi for the following call to cbfs_lookup(), rather than using the real function return value. (Why it would do that is anyone's guess because this seems unlikely to be faster than just moving the function return value from %rax into %rdi like a normal compiler.) Long story short, this optimization breaks our tests because cbfs_lookup() will be called with the wrong pointer. clang doesn't provide many options to disable individual optimizations, so the only solution seems to be to make clang aware that the function is weak during the compilation stage already, so it can be aware that it may get replaced. This patch implements that by marking the mocked functions weak via #pragma weak lines in the per-test autogenerated config header. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: I1f9011f444248544de7a71bbefc54edc006ae0cd Reviewed-on: https://review.coreboot.org/c/coreboot/+/57009 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Menzel <paulepanter@mailbox.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-08-18 04:50:22 +02:00
printf '#ifdef __clang__\n' >> $$@;
printf '#pragma clang system_header\n' >> $$@;
printf '#endif\n' >> $$@;
printf '#ifdef __TEST_SRCOBJ__\n' >> $$@;
for m in $$($(1)-mocks); do \
printf '#pragma weak %s\n' "$$$$m" >> $$@; \
done
printf '#endif\n' >> $$@;
tests: Enable config override for tests Some tests require to change kconfig symbols values to cover the code. This patch enables one to set these vaues using <test-name>-config variable. Example for integer values. timestamp-test-config += CONFIG_HAVE_MONOTONIC_TIMER=1 Example for string values. Notice escaped quotes. spd_cache-test-config += CONFIG_SPD_CACHE_FMAP_NAME=\"SPD_CACHE_FMAP_NAME\" Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I1aeb78362c2609fbefbfd91c0f58ec19ed258ee1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/52937 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-04-28 16:50:51 +02:00
$($(1)-objs): TEST_CFLAGS += -I$$(dir $$($(1)-config-file)) \
tests: Improve test output readability When running multiple tests, e.g. by using unit-tests target, it is hard to differentiate, which output comes from which file and/or configuration. This patch makes the output easier to analyze and understand by using new wrapper macro cb_run_group_tests(). This macro uses __TEST_NAME__ value (containing test path and Makefile test name) as a group name when calling cmocka group runner. Example: Test path: tests/lib/ Makefile test name: cbmem_stage_cache-test Test group array name: tests Result: tests/lib/cbmem_stage_cache-test(tests) Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I4fd936d00d77cbe2637b857ba03b4a208428ea0d Reviewed-on: https://review.coreboot.org/c/coreboot/+/57144 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-08-25 16:27:35 +02:00
-D__$$(shell echo $$($(1)-stage) | tr '[:lower:]' '[:upper:]')__ \
tests: Fix JUNIT_OUTPUT=y to write to files instead of stderr CB:57144 broke JUNIT_OUTPUT=y, and unit-tests were writing to stderr instead of selected XML files, because test name used for XML file creation contains test path. Build system did not create necessary directiories, which CMocka required to create output files. This commit fixes writing to XML files with JUNIT_OUTPUT=y, but had to sacrifice path in test name, as it was causing a lot of problems (because CMocka does not know, how to write multiple test groups to one XML file, so it uses test group name [here __TEST_NAME__(test_group)] as part of output filename). Example: Test: tests/lib/rtc-test Output file: `build/tests/junit-tests_lib_rtc-test(tests).xml Change-Id: I09891aca923bf1271cafeaa09f89b6539022709c Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/58163 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-10-07 11:06:42 +02:00
-D__TEST_NAME__=\"$(subst /,_,$(1))\"
tests/Makefile.inc: Add function wrapping mechanism This patch extends mocks functionality to allowing wrapping of mocked functions. Original function name will be prefixed with `__real_`. Example: - Mocked function: cbfs_lookup() - New function name: __real_cbfs_lookup() - Mock name: cbfs_lookup() Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I7cd0d66a17029955cbf75c8b155a7ebb7f5513aa Reviewed-on: https://review.coreboot.org/c/coreboot/+/56719 Reviewed-by: Julius Werner <jwerner@chromium.org> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2021-07-29 13:27:54 +02:00
tests: Fix function mocking for clang clang seems to like to do some aggressive optimizations that break our approach of mocking functions for test by using objcopy to turn them weak after the fact on individual compiled object files. For example, in CB:56601 the function cbfs_get_boot_device() is mocked this way. When compiling the cbfs_boot_lookup() function in src/lib/cbfs.c with clang, it will generate a normal callq instruction to a relocation for cbfs_boot_lookup(), which can then later be pointed to the mocked version of that function. However, it will also somehow infer that the version of cbfs_boot_lookup() in that file can only ever return a pointer to the static local `ro` variable (because CONFIG_VBOOT is disabled in the environment for that particular test), and instead generate instructions that directly load the address of a relocation for that variable into %rdi for the following call to cbfs_lookup(), rather than using the real function return value. (Why it would do that is anyone's guess because this seems unlikely to be faster than just moving the function return value from %rax into %rdi like a normal compiler.) Long story short, this optimization breaks our tests because cbfs_lookup() will be called with the wrong pointer. clang doesn't provide many options to disable individual optimizations, so the only solution seems to be to make clang aware that the function is weak during the compilation stage already, so it can be aware that it may get replaced. This patch implements that by marking the mocked functions weak via #pragma weak lines in the per-test autogenerated config header. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: I1f9011f444248544de7a71bbefc54edc006ae0cd Reviewed-on: https://review.coreboot.org/c/coreboot/+/57009 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Menzel <paulepanter@mailbox.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-08-18 04:50:22 +02:00
# Give us a way to distinguish between coreboot source files and test files in code.
$($(1)-srcobjs): TEST_CFLAGS += -D__TEST_SRCOBJ__
tests/Makefile.inc: Add function wrapping mechanism This patch extends mocks functionality to allowing wrapping of mocked functions. Original function name will be prefixed with `__real_`. Example: - Mocked function: cbfs_lookup() - New function name: __real_cbfs_lookup() - Mock name: cbfs_lookup() Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I7cd0d66a17029955cbf75c8b155a7ebb7f5513aa Reviewed-on: https://review.coreboot.org/c/coreboot/+/56719 Reviewed-by: Julius Werner <jwerner@chromium.org> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2021-07-29 13:27:54 +02:00
# Compile sources and apply mocking/wrapping of selected symbols.
# For each listed mock add new symbol with prefix `__real_`,
# and pointing to the same section:address.
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
$($(1)-objs): $(testobj)/$(1)/%.o: $$$$*.c $$($(1)-config-file)
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
mkdir -p $$(dir $$@)
tests/Makefile.inc: Correct dependency file suffix The dependency file, generated by $(HOSTCC) ... -MMD -MT $@ -c $< -o $@.orig is determined by replacing the suffix of "$@.orig" with ".d", resulting in "$@.d" (which is *.o.d). The file name was accidentally changed in CB:55360. Now explicitly specify the path by the "-MF" option. BUG=none TEST=make unit-tests; find build/tests/ -name "*.d" BRANCH=none Change-Id: I01f77ebaaae78dd9e69394a49e524f1013857195 Signed-off-by: Yu-Ping Wu <yupingso@chromium.org> Reviewed-on: https://review.coreboot.org/c/coreboot/+/56444 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-07-20 08:32:40 +02:00
$(HOSTCC) $(HOSTCFLAGS) $$(TEST_CFLAGS) $($(1)-cflags) -MMD \
-MF $$(basename $$@).d -MT $$@ -c $$< -o $$@.orig
tests/Makefile.inc: Add function wrapping mechanism This patch extends mocks functionality to allowing wrapping of mocked functions. Original function name will be prefixed with `__real_`. Example: - Mocked function: cbfs_lookup() - New function name: __real_cbfs_lookup() - Mock name: cbfs_lookup() Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I7cd0d66a17029955cbf75c8b155a7ebb7f5513aa Reviewed-on: https://review.coreboot.org/c/coreboot/+/56719 Reviewed-by: Julius Werner <jwerner@chromium.org> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2021-07-29 13:27:54 +02:00
objcopy_wrap_flags=''; \
for sym in $$($(1)-mocks); do \
tests: Fix function mocking for clang clang seems to like to do some aggressive optimizations that break our approach of mocking functions for test by using objcopy to turn them weak after the fact on individual compiled object files. For example, in CB:56601 the function cbfs_get_boot_device() is mocked this way. When compiling the cbfs_boot_lookup() function in src/lib/cbfs.c with clang, it will generate a normal callq instruction to a relocation for cbfs_boot_lookup(), which can then later be pointed to the mocked version of that function. However, it will also somehow infer that the version of cbfs_boot_lookup() in that file can only ever return a pointer to the static local `ro` variable (because CONFIG_VBOOT is disabled in the environment for that particular test), and instead generate instructions that directly load the address of a relocation for that variable into %rdi for the following call to cbfs_lookup(), rather than using the real function return value. (Why it would do that is anyone's guess because this seems unlikely to be faster than just moving the function return value from %rax into %rdi like a normal compiler.) Long story short, this optimization breaks our tests because cbfs_lookup() will be called with the wrong pointer. clang doesn't provide many options to disable individual optimizations, so the only solution seems to be to make clang aware that the function is weak during the compilation stage already, so it can be aware that it may get replaced. This patch implements that by marking the mocked functions weak via #pragma weak lines in the per-test autogenerated config header. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: I1f9011f444248544de7a71bbefc54edc006ae0cd Reviewed-on: https://review.coreboot.org/c/coreboot/+/57009 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Menzel <paulepanter@mailbox.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-08-18 04:50:22 +02:00
sym_line="$$$$($(OBJDUMP) -t $$@.orig | grep -E "[0-9a-fA-F]+\\s+w\\s+F\\s+.*\\s$$$$sym$$$$")"; \
tests/Makefile.inc: Add function wrapping mechanism This patch extends mocks functionality to allowing wrapping of mocked functions. Original function name will be prefixed with `__real_`. Example: - Mocked function: cbfs_lookup() - New function name: __real_cbfs_lookup() - Mock name: cbfs_lookup() Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I7cd0d66a17029955cbf75c8b155a7ebb7f5513aa Reviewed-on: https://review.coreboot.org/c/coreboot/+/56719 Reviewed-by: Julius Werner <jwerner@chromium.org> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2021-07-29 13:27:54 +02:00
if [ ! -z "$$$$sym_line" ] ; then \
addr="$$$$(echo \"$$$$sym_line\" | awk '{ print $$$$1 }')"; \
section="$$$$(echo \"$$$$sym_line\" | awk '{ print $$$$(NF - 2) }')"; \
objcopy_wrap_flags="$$$$objcopy_wrap_flags --add-symbol __real_$$$${sym}=$$$${section}:0x$$$${addr},function,global"; \
fi \
done ; \
tests: Fix function mocking for clang clang seems to like to do some aggressive optimizations that break our approach of mocking functions for test by using objcopy to turn them weak after the fact on individual compiled object files. For example, in CB:56601 the function cbfs_get_boot_device() is mocked this way. When compiling the cbfs_boot_lookup() function in src/lib/cbfs.c with clang, it will generate a normal callq instruction to a relocation for cbfs_boot_lookup(), which can then later be pointed to the mocked version of that function. However, it will also somehow infer that the version of cbfs_boot_lookup() in that file can only ever return a pointer to the static local `ro` variable (because CONFIG_VBOOT is disabled in the environment for that particular test), and instead generate instructions that directly load the address of a relocation for that variable into %rdi for the following call to cbfs_lookup(), rather than using the real function return value. (Why it would do that is anyone's guess because this seems unlikely to be faster than just moving the function return value from %rax into %rdi like a normal compiler.) Long story short, this optimization breaks our tests because cbfs_lookup() will be called with the wrong pointer. clang doesn't provide many options to disable individual optimizations, so the only solution seems to be to make clang aware that the function is weak during the compilation stage already, so it can be aware that it may get replaced. This patch implements that by marking the mocked functions weak via #pragma weak lines in the per-test autogenerated config header. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: I1f9011f444248544de7a71bbefc54edc006ae0cd Reviewed-on: https://review.coreboot.org/c/coreboot/+/57009 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Menzel <paulepanter@mailbox.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-08-18 04:50:22 +02:00
$(OBJCOPY) $$@.orig $$$$objcopy_wrap_flags $$@
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
tests: Build Cmocka from source Relying on Cmocka packages, which are provided with different OS distributions, may introduce some problems with setup environments across developers (e.g. library version mismatch). Instead, let's build Cmocka from source code, which is now added to git submodules as 3rdparty/cmocka. Please note, that cmake tool is required for building Cmocka (thus also coreboot unit tests). Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: Ia947c5c60d5c58b76acebe4b614dd427ef995950 Reviewed-on: https://review.coreboot.org/c/coreboot/+/41653 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2020-05-22 09:57:17 +02:00
$($(1)-bin): $($(1)-objs) $(CMOCKA_LIB)
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(HOSTCC) $$^ $($(1)-cflags) $$(TEST_LDFLAGS) -o $$@
endef
$(foreach test, $(alltests), \
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval $(test)-srcobjs := $(addprefix $(testobj)/$(test)/, \
tests: Rework mocking facility Using the linker's --wrap feature has the downside that it only covers references across object files: If foo.c defines a() and b(), with b calling a, --wrap=a does nothing to that call. Instead, use objcopy to mark a weak and global so it can be overridden by another implementation, but only for files originating in src/. That way mocks - implemented in tests/ - become the source of truth. TEST=Had such an issue with get_log_level() in a follow-up commit, and the mock now takes over. Also, all existing unit tests still pass. Change-Id: I99c6d6e44ecfc73366bf464d9c51c7da3f8db388 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/55360 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-06-09 18:37:42 +02:00
$(patsubst %.c,%.o,$(filter src/%,$($(test)-srcs))))) \
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval $(test)-objs := $(addprefix $(testobj)/$(test)/, \
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(patsubst %.c,%.o,$($(test)-srcs)))))
$(foreach test, $(alltests), \
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval $(test)-bin := $(testobj)/$(test)/run))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(foreach test, $(alltests), \
$(eval $(call TEST_CC_template,$(test))))
$(foreach test, $(alltests), \
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval all-test-objs += $($(test)-objs)))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(foreach test, $(alltests), \
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval test-bins += $($(test)-bin)))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
DEPENDENCIES += $(addsuffix .d,$(basename $(all-test-objs)))
-include $(DEPENDENCIES)
tests: Build Cmocka from source Relying on Cmocka packages, which are provided with different OS distributions, may introduce some problems with setup environments across developers (e.g. library version mismatch). Instead, let's build Cmocka from source code, which is now added to git submodules as 3rdparty/cmocka. Please note, that cmake tool is required for building Cmocka (thus also coreboot unit tests). Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: Ia947c5c60d5c58b76acebe4b614dd427ef995950 Reviewed-on: https://review.coreboot.org/c/coreboot/+/41653 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2020-05-22 09:57:17 +02:00
# Build cmocka
$(CMOCKA_LIB):
echo "*** Building CMOCKA ***"
mkdir -p $(cmockaobj)
cd $(cmockaobj) && $(CMAKE) $(abspath $(cmockasrc))
$(MAKE) -C $(cmockaobj)
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
# Kconfig targets
$(TEST_DOTCONFIG):
mkdir -p $(dir $@)
cp $(TEST_DEFAULT_CONFIG) $(TEST_DOTCONFIG)
# Don't override default Kconfig variables, since this will affect all
# Kconfig targets. Change them only when calling sub-make instead.
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(TEST_KCONFIG_AUTOHEADER): TEST_KCONFIG_FLAGS := DOTCONFIG=$(TEST_DOTCONFIG) \
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
KCONFIG_AUTOHEADER=$(TEST_KCONFIG_AUTOHEADER) \
KCONFIG_AUTOCONFIG=$(TEST_KCONFIG_AUTOCONFIG) \
KCONFIG_DEPENDENCIES=$(TEST_KCONFIG_DEPENDENCIES) \
KCONFIG_SPLITCONFIG=$(TEST_KCONFIG_SPLITCONFIG) \
KCONFIG_TRISTATE=$(TEST_KCONFIG_TRISTATE) \
KBUILD_DEFCONFIG=$(TEST_DEFAULT_CONFIG)
$(TEST_KCONFIG_AUTOHEADER): $(TEST_DOTCONFIG) $(objutil)/kconfig/conf
mkdir -p $(dir $@)
util/kconfig: Uprev to Linux 5.13's kconfig This was originally several commits that had to be squashed into one because the intermediate states weren't able to build coreboot: - one to remove everything that wasn't our own code, leaving only regex.[ch], toada.c, description.md and Makefile.inc. - one to copy in Linux 5.13's scripts/kconfig and adapt Makefile.inc to make the original Makefile work again. - adapt abuild to use olddefconfig, simplifying matters. - apply patches in util/kconfig/patches. - Some more adaptations to the libpayload build system. The patches are now in util/kconfig/patches/, reverse applying them should lead to a util/kconfig/ tree that contains exactly the Linux version + our own 5 files. Change-Id: Ia0e8fe4e9022b278f34ab113a433ef4d45e5c355 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/37152 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Raul Rangel <rrangel@chromium.org>
2019-11-22 16:55:58 +01:00
$(MAKE) $(TEST_KCONFIG_FLAGS) olddefconfig
$(MAKE) $(TEST_KCONFIG_FLAGS) syncconfig
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(TEST_KCONFIG_AUTOCONFIG): $(TEST_KCONFIG_AUTOHEADER)
true
.PHONY: $(alltests) $(addprefix clean-,$(alltests))
.PHONY: unit-tests build-unit-tests run-unit-tests clean-unit-tests
tests: Fix JUNIT_OUTPUT=y to write to files instead of stderr CB:57144 broke JUNIT_OUTPUT=y, and unit-tests were writing to stderr instead of selected XML files, because test name used for XML file creation contains test path. Build system did not create necessary directiories, which CMocka required to create output files. This commit fixes writing to XML files with JUNIT_OUTPUT=y, but had to sacrifice path in test name, as it was causing a lot of problems (because CMocka does not know, how to write multiple test groups to one XML file, so it uses test group name [here __TEST_NAME__(test_group)] as part of output filename). Example: Test: tests/lib/rtc-test Output file: `build/tests/junit-tests_lib_rtc-test(tests).xml Change-Id: I09891aca923bf1271cafeaa09f89b6539022709c Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/58163 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-10-07 11:06:42 +02:00
# %g in CMOCKA_XML_FILE will be replaced with "__TEST_NAME__(<test-group-name>)"
# by macro cb_run_group_tests(), which should be used for running tests.
# __TEST_NAME__ contains test name including path e.g. tests_lib_rtc-test
tests: Allow emitting junit output for unit tests Change-Id: Iab0c4250b1baa77d4eab7538ec1fd3310f9e63e4 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/41771 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Martin Roth <martinroth@google.com> Reviewed-by: Angel Pons <th3fanbus@gmail.com> Reviewed-by: Jan Dabros <jsd@semihalf.com>
2020-05-27 10:59:09 +02:00
ifeq ($(JUNIT_OUTPUT),y)
$(alltests): export CMOCKA_MESSAGE_OUTPUT=xml
tests: Fix JUNIT_OUTPUT=y to write to files instead of stderr CB:57144 broke JUNIT_OUTPUT=y, and unit-tests were writing to stderr instead of selected XML files, because test name used for XML file creation contains test path. Build system did not create necessary directiories, which CMocka required to create output files. This commit fixes writing to XML files with JUNIT_OUTPUT=y, but had to sacrifice path in test name, as it was causing a lot of problems (because CMocka does not know, how to write multiple test groups to one XML file, so it uses test group name [here __TEST_NAME__(test_group)] as part of output filename). Example: Test: tests/lib/rtc-test Output file: `build/tests/junit-tests_lib_rtc-test(tests).xml Change-Id: I09891aca923bf1271cafeaa09f89b6539022709c Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/58163 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-10-07 11:06:42 +02:00
$(alltests): export CMOCKA_XML_FILE=$(testobj)/junit-%g.xml
tests: Allow emitting junit output for unit tests Change-Id: Iab0c4250b1baa77d4eab7538ec1fd3310f9e63e4 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/41771 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Martin Roth <martinroth@google.com> Reviewed-by: Angel Pons <th3fanbus@gmail.com> Reviewed-by: Jan Dabros <jsd@semihalf.com>
2020-05-27 10:59:09 +02:00
endif
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(alltests): $$($$(@)-bin)
tests: Fix JUNIT_OUTPUT=y to write to files instead of stderr CB:57144 broke JUNIT_OUTPUT=y, and unit-tests were writing to stderr instead of selected XML files, because test name used for XML file creation contains test path. Build system did not create necessary directiories, which CMocka required to create output files. This commit fixes writing to XML files with JUNIT_OUTPUT=y, but had to sacrifice path in test name, as it was causing a lot of problems (because CMocka does not know, how to write multiple test groups to one XML file, so it uses test group name [here __TEST_NAME__(test_group)] as part of output filename). Example: Test: tests/lib/rtc-test Output file: `build/tests/junit-tests_lib_rtc-test(tests).xml Change-Id: I09891aca923bf1271cafeaa09f89b6539022709c Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/58163 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-10-07 11:06:42 +02:00
rm -f $(testobj)/junit-$(subst /,_,$(patsubst $(testobj)/%/,%,$(dir $^)))\(*\).xml
rm -f $(testobj)/$(subst /,_,$^).failed
tests: Always run all unit tests So far, the semantics have been that run-unit-tests stopped at the first test suite that failed. This hides useful signal in later tests, so always run all tests and collect the result. Change-Id: I407715f85513c2c95a1cf89cfb427317dff9fbab Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/41773 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jan Dabros <jsd@semihalf.com> Reviewed-by: Martin Roth <martinroth@google.com> Reviewed-by: Angel Pons <th3fanbus@gmail.com>
2020-05-27 11:39:32 +02:00
-./$^ || echo failed > $(testobj)/$(subst /,_,$^).failed
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
# Build a code coverage report by collecting all the gcov files into a single
# report. If COV is not set, this might be a user error, and they're trying
# to generate a coverage report without first having built and run the code
# with code coverage. So instead of silently correcting it by adding COV=1,
# let's flag it to the user so they can be sure they're doing the thing they
# want to do.
tests: enable code coverage for unit tests Add a new `coverage-unit-tests` make target that builds the unit tests for code coverage, runs the tests, and generates a coverage report. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I6ea780ee9e246c0bb8c35b8e0de4252431dabbff Reviewed-on: https://review.coreboot.org/c/coreboot/+/52444 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-04-16 19:36:25 +02:00
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
.PHONY: coverage-report clean-coverage-report
ifeq ($(COV),1)
coverage-report:
lcov -o $(testobj)/tests.info -c -d $(testobj) --exclude '$(testsrc)/*'
genhtml -q -o $(testobj)/$(coverage_dir) -t "coreboot unit tests" \
tests: enable code coverage for unit tests Add a new `coverage-unit-tests` make target that builds the unit tests for code coverage, runs the tests, and generates a coverage report. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I6ea780ee9e246c0bb8c35b8e0de4252431dabbff Reviewed-on: https://review.coreboot.org/c/coreboot/+/52444 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-04-16 19:36:25 +02:00
-s $(testobj)/tests.info
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
clean-coverage-report:
rm -Rf $(testobj)/$(coverage_dir)
else
coverage-report:
COV=1 V=$(V) $(MAKE) coverage-report
clean-coverage-report:
COV=1 V=$(V) $(MAKE) clean-coverage-report
endif
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
unit-tests: build-unit-tests run-unit-tests
build-unit-tests: $(test-bins)
run-unit-tests: $(alltests)
tests: Always run all unit tests So far, the semantics have been that run-unit-tests stopped at the first test suite that failed. This hides useful signal in later tests, so always run all tests and collect the result. Change-Id: I407715f85513c2c95a1cf89cfb427317dff9fbab Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/41773 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jan Dabros <jsd@semihalf.com> Reviewed-by: Martin Roth <martinroth@google.com> Reviewed-by: Angel Pons <th3fanbus@gmail.com>
2020-05-27 11:39:32 +02:00
if [ `find $(testobj) -name '*.failed' | wc -l` -gt 0 ]; then \
echo "**********************"; \
echo " TESTS FAILED"; \
echo "**********************"; \
exit 1; \
else \
echo "**********************"; \
echo " ALL TESTS PASSED"; \
echo "**********************"; \
exit 0; \
fi
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(addprefix clean-,$(alltests)): clean-%:
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
rm -rf $(testobj)/$*
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
clean-unit-tests:
rm -rf $(testobj)
Makefile: Add unit-tests help and targets list Add unit-tests targets to help output. Add list-unit-tests target that lists all available unit-tests. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I464a76cbea1f4afbc3fc772960787952e61b95b9 Reviewed-on: https://review.coreboot.org/c/coreboot/+/52293 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Patrick Georgi <pgeorgi@google.com> Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-04-13 16:07:05 +02:00
list-unit-tests:
@echo "unit-tests:"
for t in $(sort $(alltests)); do \
echo " $$t"; \
done
help-unit-tests help::
@echo '*** coreboot unit-tests targets ***'
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
@echo ' Use "COV=1 make [target]" to enable code coverage for unit tests'
tests: enable code coverage for unit tests Add a new `coverage-unit-tests` make target that builds the unit tests for code coverage, runs the tests, and generates a coverage report. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I6ea780ee9e246c0bb8c35b8e0de4252431dabbff Reviewed-on: https://review.coreboot.org/c/coreboot/+/52444 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-04-16 19:36:25 +02:00
@echo ' unit-tests - Run all unit-tests from tests/'
@echo ' clean-unit-tests - Remove unit-tests build artifacts'
@echo ' list-unit-tests - List all unit-tests'
@echo ' <unit-test> - Build and run single unit-test'
@echo ' clean-<unit-test> - Remove single unit-test build artifacts'
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
@echo ' coverage-report - Generate a code coverage report'
@echo ' clean-coverage-report - Remove the code coverage report'
Makefile: Add unit-tests help and targets list Add unit-tests targets to help output. Add list-unit-tests target that lists all available unit-tests. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I464a76cbea1f4afbc3fc772960787952e61b95b9 Reviewed-on: https://review.coreboot.org/c/coreboot/+/52293 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Patrick Georgi <pgeorgi@google.com> Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-04-13 16:07:05 +02:00
@echo