GNUBoot
/
coreboot-kgpe-d16
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
coreboot-kgpe-d16/tests/Makefile.mk

197 lines
6.4 KiB
Makefile
Raw Normal View History

tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
# SPDX-License-Identifier: GPL-2.0-only
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
# Place the build output in one of two places depending on COV, so that code
# built with code coverage never mixes with code built without code coverage.
ifeq ($(COV),1)
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
testobj := $(obj)/coverage
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
else
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
testobj := $(obj)/tests
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
endif
tests/Makefile.inc: Build utilities in separate directory Utilities like kconfig/conf now will be built inside tests build tree. It will eliminate possible colisions of target names when using test framework in more than one place (see CB:70110) Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I4c1eb901c921f4ec6ee8985b154362153c5fd0e7 Reviewed-on: https://review.coreboot.org/c/coreboot/+/70359 Reviewed-by: Jan Dabros <jsd@semihalf.com> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-12-05 15:21:52 +01:00
objutil := $(testobj)/util
tests: Split Makefile to allow for making host-side test tools This patch is based on similar changes [1] done in Depthcharge projects, which aimed to provide unified way to build host-side programs for testing internal code. New test tools might benefit from it by having same base code as unit-tests. [1] https://crrev.com/c/3412108 TEST=make unit-tests TEST=COV=1 make unit-tests coverage-report Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Iac4517ab6146fa3f2d2b7a20df54601ab2d04c3d Reviewed-on: https://review.coreboot.org/c/coreboot/+/63637 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Arthur Heymans <arthur@aheymans.xyz>
2022-04-14 13:56:02 +02:00
include $(top)/tests/Makefile.common
rmodtool: Make memlayout symbols absolute and do not relocate them Memlayout is a mechanism to define memory areas outside the normal program segment constructed by the linker. Therefore, it generally doesn't make sense to relocate memlayout symbols when the program is relocated. They tend to refer to things that are always in one specific spot, independent of where the program is loaded. This hasn't really hurt us in the past because the use case we have for rmodules (ramstage on x86) just happens to not really need to refer to any memlayout-defined areas at the moment. But that use case may come up in the future so it's still worth fixing. This patch declares all memlayout-defined symbols as ABSOLUTE() in the linker, which is then reflected in the symbol table of the generated ELF. We can then use that distinction to have rmodtool skip them when generating the relocation table for an rmodule. (Also rearrange rmodtool a little to make the primary string table more easily accessible to the rest of the code, so we can refer to symbol names in debug output.) A similar problem can come up with userspace unit tests, but we cannot modify the userspace relocation toolchain (and for unfortunate historical reasons, it tries to relocate even absolute symbols). We'll just disable PIC and make those binaries fully static to avoid that issue. Signed-off-by: Julius Werner <jwerner@chromium.org> Change-Id: Ic51d9add3dc463495282b365c1b6d4a9bf11dbf2 Reviewed-on: https://review.coreboot.org/c/coreboot/+/50629 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2021-02-13 02:37:27 +01:00
tests: Add option for debug symbols & no optimization To make it easier to build the tests with debug symbols, add a check for the "GDB_DEBUG" environment variable. If set, build with -g and -Og to enable the symbols and disable optimization. Signed-off-by: Martin Roth <gaumless@gmail.com> Change-Id: I3a644dcccb7e15473413b775da8f70617afaefce Reviewed-on: https://review.coreboot.org/c/coreboot/+/67918 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2022-09-28 02:30:55 +02:00
# Enable GDB debug build if requested
GDB_DEBUG ?= 0
ifneq ($(GDB_DEBUG),0)
TEST_CFLAGS += -g -Og
endif
# Enable code coverage if requested
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
ifeq ($(COV),1)
TEST_CFLAGS += --coverage
TEST_LDFLAGS += --coverage
endif
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
stages := decompressor bootblock romstage smm verstage
stages += ramstage rmodule postcar libagesa
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
alltests :=
subdirs := tests/arch tests/acpi tests/commonlib tests/console tests/cpu
drivers/efi: Add EFI variable store option support Add a driver to read and write EFI variables stored in a region device. This is particularly useful for EDK2 as payload and allows to reuse existing EFI tools to set/get options used by the firmware. The write implementation is fault tolerant and doesn't corrupt the variable store. A faulting write might result in using the old value even though a 'newer' had been completely written. Implemented basic unit tests for header corruption, writing existing data and append new data into the store. Initial firmware region state: Initially the variable store region isn't formatted. Usually this is done in the EDK2 payload when no valid firmware volume could be found. It might be useful to do this offline or in coreboot to have a working option store on the first boot or when it was corrupted. Performance improvements: Right now the code always checks if the firmware volume header is valid. This could be optimised by caching the test result in heap. For write operations it would be good to cache the end of the variable store in the heap as well, instead of walking the whole store. For read operations caching the entire store could be considered. Reclaiming memory: The EFI variable store is append write only. To update an existing variable, first a new is written to the end of the store and then the previous is marked invalid. This only works on PNOR flash that allow to clear set bits, but keep cleared bits state. This mechanisms allows a fault tolerant write, but it also requires to "clean" the variable store for time to time. This cleaning would remove variables that have been marked "deleted". Such cleaning mechanism in turn must be fault tolerant and thus must use a second partition in the SPI flash as backup/working region. For now to cleaning is done in coreboot. Fault checking: The driver should check if a previous write was successful and if not mark variables as deleted on the next operation. Tested and working: - Enumerate all existing variables - Read variables - Write variables Change-Id: I8079f71d29da5dc2db956fc68bef1486fe3906bb Signed-off-by: Patrick Rudolph <patrick.rudolph@9elements.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/52564 Reviewed-by: Lean Sheng Tan <sheng.tan@9elements.com> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2021-04-21 10:02:55 +02:00
subdirs += tests/device tests/drivers tests/ec tests/lib
subdirs += tests/mainboard tests/northbridge tests/security tests/soc
subdirs += tests/southbridge tests/superio tests/vendorcode
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
define tests-handler
alltests += $(1)$(2)
$(foreach attribute,$(attributes),
$(eval $(1)$(2)-$(attribute) += $($(2)-$(attribute))))
$(foreach attribute,$(attributes),
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval $(2)-$(attribute) := ))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
# Sanity check for stage attribute value
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval $(1)$(2)-stage := $(if $($(1)$(2)-stage),$($(1)$(2)-stage),ramstage))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(if $(findstring $($(1)$(2)-stage), $(stages)),,
$(error Wrong $(1)$(2)-stage value $($(1)$(2)-stage). \
tests: Rename Makefiles from .inc to .mk The .inc suffix is confusing to various tools as it's not specific to Makefiles. This means that editors don't recognize the files, and don't open them with highlighting and any other specific editor functionality. This issue is also seen in the release notes generation script where Makefiles get renamed before running cloc. Signed-off-by: Martin Roth <gaumless@gmail.com> Change-Id: Id09eafd293a54198aab87281f529749325df8b07 Reviewed-on: https://review.coreboot.org/c/coreboot/+/80122 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Felix Held <felix-coreboot@felixheld.de> Reviewed-by: Felix Singer <service+coreboot-gerrit@felixsinger.de>
2024-01-19 02:16:12 +01:00
Check your $(dir $(1)$(2))Makefile.mk))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
endef
$(call add-special-class, tests)
$(call evaluate_subdirs)
$(foreach test, $(alltests), \
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval $(test)-srcobjs := $(addprefix $(testobj)/$(test)/, \
tests: Rework mocking facility Using the linker's --wrap feature has the downside that it only covers references across object files: If foo.c defines a() and b(), with b calling a, --wrap=a does nothing to that call. Instead, use objcopy to mark a weak and global so it can be overridden by another implementation, but only for files originating in src/. That way mocks - implemented in tests/ - become the source of truth. TEST=Had such an issue with get_log_level() in a follow-up commit, and the mock now takes over. Also, all existing unit tests still pass. Change-Id: I99c6d6e44ecfc73366bf464d9c51c7da3f8db388 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/55360 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-06-09 18:37:42 +02:00
$(patsubst %.c,%.o,$(filter src/%,$($(test)-srcs))))) \
tests: Support linking with system libc This patch allows for linking selected files with system libc. This allows for creating libraries interacting with filesystem, standard I/O and other parts of system. Until now it was only possible using CMocka proxy functions or functions not masked by code under test. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I652362ba61a25e974d706357fc36479ccee763e4 Reviewed-on: https://review.coreboot.org/c/coreboot/+/70108 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jan Dabros <jsd@semihalf.com>
2022-11-14 11:22:59 +01:00
$(eval $(test)-sysobjs := $(addprefix $(testobj)/$(test)/, \
$(patsubst %.c,%.o,$($(test)-syssrcs)))) \
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval $(test)-objs := $(addprefix $(testobj)/$(test)/, \
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(patsubst %.c,%.o,$($(test)-srcs)))))
$(foreach test, $(alltests), \
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval $(test)-bin := $(testobj)/$(test)/run))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(foreach test, $(alltests), \
$(eval $(call TEST_CC_template,$(test))))
$(foreach test, $(alltests), \
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval all-test-objs += $($(test)-objs)))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(foreach test, $(alltests), \
tests/Makefile.inc: Change assignments to immediate and fix style Change assignments to be immediate instead of lazy. Add spaces between variable name and assignment operators to increase readability. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: Idf07b5a836b33cd80c3533e582e2a1916a7bd45d Reviewed-on: https://review.coreboot.org/c/coreboot/+/56931 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-08-09 14:20:28 +02:00
$(eval test-bins += $($(test)-bin)))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
DEPENDENCIES += $(addsuffix .d,$(basename $(all-test-objs)))
-include $(DEPENDENCIES)
tests: Add support for tests build failures detection This patch introduces new target: junit.xml-unit-tests, which builds and runs unit-tests. It also creates build log containing build logs. This feature allows for one to see build failures in Jenkins dashboard. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I94184379dcc2ac10f1a47f4a9d205cacbeb640fe Reviewed-on: https://review.coreboot.org/c/coreboot/+/67372 Reviewed-by: Julius Werner <jwerner@chromium.org> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-09-06 10:42:16 +02:00
.PHONY: $(alltests) $(addprefix clean-,$(alltests)) $(addprefix try-,$(alltests))
.PHONY: $(addprefix build-,$(alltests)) $(addprefix run-,$(alltests))
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
.PHONY: unit-tests build-unit-tests run-unit-tests clean-unit-tests
tests: Add support for tests build failures detection This patch introduces new target: junit.xml-unit-tests, which builds and runs unit-tests. It also creates build log containing build logs. This feature allows for one to see build failures in Jenkins dashboard. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I94184379dcc2ac10f1a47f4a9d205cacbeb640fe Reviewed-on: https://review.coreboot.org/c/coreboot/+/67372 Reviewed-by: Julius Werner <jwerner@chromium.org> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-09-06 10:42:16 +02:00
.PHONY: junit.xml-unit-tests clean-junit.xml-unit-tests
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
tests: Fix JUNIT_OUTPUT=y to write to files instead of stderr CB:57144 broke JUNIT_OUTPUT=y, and unit-tests were writing to stderr instead of selected XML files, because test name used for XML file creation contains test path. Build system did not create necessary directiories, which CMocka required to create output files. This commit fixes writing to XML files with JUNIT_OUTPUT=y, but had to sacrifice path in test name, as it was causing a lot of problems (because CMocka does not know, how to write multiple test groups to one XML file, so it uses test group name [here __TEST_NAME__(test_group)] as part of output filename). Example: Test: tests/lib/rtc-test Output file: `build/tests/junit-tests_lib_rtc-test(tests).xml Change-Id: I09891aca923bf1271cafeaa09f89b6539022709c Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/58163 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-10-07 11:06:42 +02:00
# %g in CMOCKA_XML_FILE will be replaced with "__TEST_NAME__(<test-group-name>)"
# by macro cb_run_group_tests(), which should be used for running tests.
# __TEST_NAME__ contains test name including path e.g. tests_lib_rtc-test
tests: Allow emitting junit output for unit tests Change-Id: Iab0c4250b1baa77d4eab7538ec1fd3310f9e63e4 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/41771 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Martin Roth <martinroth@google.com> Reviewed-by: Angel Pons <th3fanbus@gmail.com> Reviewed-by: Jan Dabros <jsd@semihalf.com>
2020-05-27 10:59:09 +02:00
ifeq ($(JUNIT_OUTPUT),y)
tests: Add support for tests build failures detection This patch introduces new target: junit.xml-unit-tests, which builds and runs unit-tests. It also creates build log containing build logs. This feature allows for one to see build failures in Jenkins dashboard. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I94184379dcc2ac10f1a47f4a9d205cacbeb640fe Reviewed-on: https://review.coreboot.org/c/coreboot/+/67372 Reviewed-by: Julius Werner <jwerner@chromium.org> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-09-06 10:42:16 +02:00
$(addprefix run-,$(alltests)): export CMOCKA_MESSAGE_OUTPUT=xml
$(addprefix run-,$(alltests)): export CMOCKA_XML_FILE=$(testobj)/junit-%g.xml
tests: Allow emitting junit output for unit tests Change-Id: Iab0c4250b1baa77d4eab7538ec1fd3310f9e63e4 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/41771 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Martin Roth <martinroth@google.com> Reviewed-by: Angel Pons <th3fanbus@gmail.com> Reviewed-by: Jan Dabros <jsd@semihalf.com>
2020-05-27 10:59:09 +02:00
endif
tests: Add support for tests build failures detection This patch introduces new target: junit.xml-unit-tests, which builds and runs unit-tests. It also creates build log containing build logs. This feature allows for one to see build failures in Jenkins dashboard. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I94184379dcc2ac10f1a47f4a9d205cacbeb640fe Reviewed-on: https://review.coreboot.org/c/coreboot/+/67372 Reviewed-by: Julius Werner <jwerner@chromium.org> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-09-06 10:42:16 +02:00
$(addprefix run-,$(alltests)): run-%: $$(%-bin)
tests: Fix JUNIT_OUTPUT=y to write to files instead of stderr CB:57144 broke JUNIT_OUTPUT=y, and unit-tests were writing to stderr instead of selected XML files, because test name used for XML file creation contains test path. Build system did not create necessary directiories, which CMocka required to create output files. This commit fixes writing to XML files with JUNIT_OUTPUT=y, but had to sacrifice path in test name, as it was causing a lot of problems (because CMocka does not know, how to write multiple test groups to one XML file, so it uses test group name [here __TEST_NAME__(test_group)] as part of output filename). Example: Test: tests/lib/rtc-test Output file: `build/tests/junit-tests_lib_rtc-test(tests).xml Change-Id: I09891aca923bf1271cafeaa09f89b6539022709c Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/58163 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org>
2021-10-07 11:06:42 +02:00
rm -f $(testobj)/junit-$(subst /,_,$(patsubst $(testobj)/%/,%,$(dir $^)))\(*\).xml
rm -f $(testobj)/$(subst /,_,$^).failed
tests/Makefile: Remove ./ prefix when running tests If ran with obj=/absolute path, then tests were failing to execute because the recipe tried running `.//absolutepath/...run`. Signed-off-by: Raul E Rangel <rrangel@chromium.org> Change-Id: I9c3638b1af7531dbe8e956dcbe168250a235ead4 Reviewed-on: https://review.coreboot.org/c/coreboot/+/58806 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-07-24 00:43:18 +02:00
-$^ || echo failed > $(testobj)/$(subst /,_,$^).failed
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
tests: Add support for tests build failures detection This patch introduces new target: junit.xml-unit-tests, which builds and runs unit-tests. It also creates build log containing build logs. This feature allows for one to see build failures in Jenkins dashboard. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I94184379dcc2ac10f1a47f4a9d205cacbeb640fe Reviewed-on: https://review.coreboot.org/c/coreboot/+/67372 Reviewed-by: Julius Werner <jwerner@chromium.org> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2022-09-06 10:42:16 +02:00
$(addprefix build-,$(alltests)): build-%: $$(%-bin)
$(alltests): run-$$(@)
$(addprefix try-,$(alltests)): try-%: clean-% $(TEST_COMMON_DEPENDENCIES)
mkdir -p $(testobj)/$*
echo "<testcase classname='coreboot_build_unit_test' name='$*'>" >> $(testobj)/$*.tmp; \
$(MAKE) V=$(V) Q=$(Q) COV=$(COV) JUNIT_OUTPUT=y "build-$*" >> $(testobj)/$*.tmp.2 2>&1 \
&& type="system-out" || type="failure"; \
if [ $$type = "failure" ]; then \
echo "<failure type='buildFailed'>" >> $(testobj)/$*.tmp; \
else \
echo "<$$type>" >> $(testobj)/$*.tmp; \
fi; \
echo '<![CDATA[' >> $(testobj)/$*.tmp; \
cat $(testobj)/$*.tmp.2 >> $(testobj)/$*.tmp; \
echo "]]></$$type>" >> $(testobj)/$*.tmp; \
rm -f $(testobj)/$*.tmp.2; \
echo "</testcase>" >> $(testobj)/$*.tmp; \
if [ $$type != 'failure' ]; then \
$(MAKE) V=$(V) Q=$(Q) COV=$(COV) JUNIT_OUTPUT=y "run-$*"; \
fi
TESTS_BUILD_XML_FILE := $(testobj)/junit-tests-build.xml
$(TESTS_BUILD_XML_FILE): clean-junit.xml-unit-tests $(addprefix try-,$(alltests))
mkdir -p $(dir $@)
echo '<?xml version="1.0" encoding="utf-8"?><testsuite>' > $@
for tst in $(alltests); do \
cat $(testobj)/$$tst.tmp >> $@; \
done
echo "</testsuite>" >> $@
junit.xml-unit-tests: $(TESTS_BUILD_XML_FILE)
clean-junit.xml-unit-tests:
rm -f $(TESTS_BUILD_XML_FILE)
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
# Build a code coverage report by collecting all the gcov files into a single
# report. If COV is not set, this might be a user error, and they're trying
# to generate a coverage report without first having built and run the code
# with code coverage. So instead of silently correcting it by adding COV=1,
# let's flag it to the user so they can be sure they're doing the thing they
# want to do.
tests: enable code coverage for unit tests Add a new `coverage-unit-tests` make target that builds the unit tests for code coverage, runs the tests, and generates a coverage report. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I6ea780ee9e246c0bb8c35b8e0de4252431dabbff Reviewed-on: https://review.coreboot.org/c/coreboot/+/52444 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-04-16 19:36:25 +02:00
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
.PHONY: coverage-report clean-coverage-report
ifeq ($(COV),1)
coverage-report:
lcov -o $(testobj)/tests.info -c -d $(testobj) --exclude '$(testsrc)/*'
genhtml -q -o $(testobj)/$(coverage_dir) -t "coreboot unit tests" \
tests: enable code coverage for unit tests Add a new `coverage-unit-tests` make target that builds the unit tests for code coverage, runs the tests, and generates a coverage report. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I6ea780ee9e246c0bb8c35b8e0de4252431dabbff Reviewed-on: https://review.coreboot.org/c/coreboot/+/52444 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-04-16 19:36:25 +02:00
-s $(testobj)/tests.info
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
clean-coverage-report:
rm -Rf $(testobj)/$(coverage_dir)
else
coverage-report:
COV=1 V=$(V) $(MAKE) coverage-report
clean-coverage-report:
COV=1 V=$(V) $(MAKE) clean-coverage-report
endif
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
unit-tests: build-unit-tests run-unit-tests
build-unit-tests: $(test-bins)
run-unit-tests: $(alltests)
tests: Always run all unit tests So far, the semantics have been that run-unit-tests stopped at the first test suite that failed. This hides useful signal in later tests, so always run all tests and collect the result. Change-Id: I407715f85513c2c95a1cf89cfb427317dff9fbab Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/41773 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jan Dabros <jsd@semihalf.com> Reviewed-by: Martin Roth <martinroth@google.com> Reviewed-by: Angel Pons <th3fanbus@gmail.com>
2020-05-27 11:39:32 +02:00
if [ `find $(testobj) -name '*.failed' | wc -l` -gt 0 ]; then \
echo "**********************"; \
echo " TESTS FAILED"; \
echo "**********************"; \
exit 1; \
else \
echo "**********************"; \
echo " ALL TESTS PASSED"; \
echo "**********************"; \
exit 0; \
fi
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
$(addprefix clean-,$(alltests)): clean-%:
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
rm -rf $(testobj)/$*
tests: Add build subsystem for unit testing coreboot Add a subsystem which will be used for writing, building and running unit tests for different coreboot's modules. This work is built using Cmocka unit testing framework. Description of what unit testing means (for the author) and how unit testing framework evaluation was performed may be found in Documentation/technotes/2020-03-unit-testing-coreboot.md Makefiles structure is very similar to this used for building coreboot images. Every directory has its own Makefile.inc were tests' names, sources, subdirs and multiple other test-related attributes are defined in form of variables. Signed-off-by: Jan Dabros <jsd@semihalf.com> Change-Id: I9b0220b84b9a6e448476ca3eb3ccccc5fb829ad1 Reviewed-on: https://review.coreboot.org/c/coreboot/+/39894 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Fagerburg <pfagerburg@chromium.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2020-03-28 00:15:03 +01:00
clean-unit-tests:
rm -rf $(testobj)
Makefile: Add unit-tests help and targets list Add unit-tests targets to help output. Add list-unit-tests target that lists all available unit-tests. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I464a76cbea1f4afbc3fc772960787952e61b95b9 Reviewed-on: https://review.coreboot.org/c/coreboot/+/52293 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Patrick Georgi <pgeorgi@google.com> Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-04-13 16:07:05 +02:00
list-unit-tests:
@echo "unit-tests:"
for t in $(sort $(alltests)); do \
echo " $$t"; \
done
help-unit-tests help::
@echo '*** coreboot unit-tests targets ***'
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
@echo ' Use "COV=1 make [target]" to enable code coverage for unit tests'
tests: Add option for debug symbols & no optimization To make it easier to build the tests with debug symbols, add a check for the "GDB_DEBUG" environment variable. If set, build with -g and -Og to enable the symbols and disable optimization. Signed-off-by: Martin Roth <gaumless@gmail.com> Change-Id: I3a644dcccb7e15473413b775da8f70617afaefce Reviewed-on: https://review.coreboot.org/c/coreboot/+/67918 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2022-09-28 02:30:55 +02:00
@echo ' Use "GDB_DEBUG=1 make [target]" to build with debug symbols'
tests: enable code coverage for unit tests Add a new `coverage-unit-tests` make target that builds the unit tests for code coverage, runs the tests, and generates a coverage report. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I6ea780ee9e246c0bb8c35b8e0de4252431dabbff Reviewed-on: https://review.coreboot.org/c/coreboot/+/52444 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-04-16 19:36:25 +02:00
@echo ' unit-tests - Run all unit-tests from tests/'
@echo ' clean-unit-tests - Remove unit-tests build artifacts'
@echo ' list-unit-tests - List all unit-tests'
@echo ' <unit-test> - Build and run single unit-test'
@echo ' clean-<unit-test> - Remove single unit-test build artifacts'
tests: improve code coverage support Fix the exclusion path for lcov; it should exclude the directory with source code, not object files. Use the COV environment variable to * control whether we build for coverage or not * select the output directory Add a separate target for generating the report, so we can get a report for all of the tests together or just a single test. Add documentation. Signed-off-by: Paul Fagerburg <pfagerburg@google.com> Change-Id: I2bd2bfdedfab291aabeaa968c10b17e9b61c9c0a Reviewed-on: https://review.coreboot.org/c/coreboot/+/54072 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Jakub Czapiga <jacz@semihalf.com>
2021-05-11 17:56:48 +02:00
@echo ' coverage-report - Generate a code coverage report'
@echo ' clean-coverage-report - Remove the code coverage report'
Makefile: Add unit-tests help and targets list Add unit-tests targets to help output. Add list-unit-tests target that lists all available unit-tests. Signed-off-by: Jakub Czapiga <jacz@semihalf.com> Change-Id: I464a76cbea1f4afbc3fc772960787952e61b95b9 Reviewed-on: https://review.coreboot.org/c/coreboot/+/52293 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Patrick Georgi <pgeorgi@google.com> Reviewed-by: Julius Werner <jwerner@chromium.org>
2021-04-13 16:07:05 +02:00
@echo