Commit Graph

51 Commits

Author SHA1 Message Date
Patrick Georgi c88d16baaf util/cbfstool: Add cbfs-compression-tool
cbfs-compression-tool provides a way to benchmark the compression
algorithms as used by cbfstool (and coreboot) and allows to
pre-compress data for later consumption by cbfstool (once it supports
the format).

For an impression, the benchmark's results on my machine:

measuring 'none'
compressing 10485760 bytes to 10485760 took 0 seconds
measuring 'LZMA'
compressing 10485760 bytes to 1736 took 2 seconds
measuring 'LZ4'
compressing 10485760 bytes to 41880 took 0 seconds

And a possible use for external compression, parallel and non-parallel
(60MB in 53 files compressed to 650KB on a machine with 40 threads):

$ time (ls -1 *.* |xargs -n 1 -P $(nproc) -I '{}' cbfs-compression-tool compress '{}' out/'{}' LZMA)

real	0m0.786s
user	0m11.440s
sys	0m0.044s

$ time (ls -1 *.* |xargs -n 1 -P 1 -I '{}' cbfs-compression-tool compress '{}' out/'{}' LZMA)

real	0m10.444s
user	0m10.280s
sys	0m0.064s

Change-Id: I40be087e85d09a895b1ed277270350ab65a4d6d4
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Reviewed-on: https://review.coreboot.org/18099
Tested-by: build bot (Jenkins)
Reviewed-by: Martin Roth <martinroth@google.com>
2017-01-12 21:40:25 +01:00
Patrick Georgi 8474e7d7e8 util/cbfstool: compile with -O2 by default
This speeds up the lzma encoder approximately four-fold.

Change-Id: Ibf896098799693ddd0f8a6c74bda2e518ecea869
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Reviewed-on: https://review.coreboot.org/18098
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2017-01-12 21:40:00 +01:00
Paul Kocialkowski 61486b506d Use VBOOT_SOURCE instead of hardcoding vboot path
This replaces all occurrences of a hardcoded vboot path to the
VBOOT_SOURCE variable, that may be overridden from the command line,
witch fallback to the source from 3rdparty.

Change-Id: Ia57d498d38719cc71e17060b76b0162c4ab363ed
Signed-off-by: Paul Kocialkowski <contact@paulk.fr>
Reviewed-on: https://review.coreboot.org/15825
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
Reviewed-by: Julius Werner <jwerner@chromium.org>
2016-08-08 12:17:00 +02:00
Patrick Georgi 2b80734811 fmaptool: Accept hex values with uppercase letters
Due to a newer flex version with which the scanner was recreated, we
also have to make the compiler less strict on the generated code.

Change-Id: I3758c0dcb2f5661d072b54a30d6a4ebe094854e6
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Reviewed-on: https://review.coreboot.org/15482
Tested-by: build bot (Jenkins)
Reviewed-by: Werner Zeh <werner.zeh@siemens.com>
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2016-07-04 12:17:51 +02:00
Furquan Shaikh 233f1b6a38 ifwitool: Add new tool for managing IFWI images
- Supports following operations:
 1. add raw/dir sub-partition
 2. extract raw/dir sub-partition
 3. print info
 4. delete raw sub-partition
 5. replace raw/dir sub-partition

Change-Id: I683a0ab13cc50eb60eecca34db4a8ffefc8dccbd
Signed-off-by: Furquan Shaikh <furquan@google.com>
Reviewed-on: https://review.coreboot.org/14896
Tested-by: build bot (Jenkins)
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-05-30 23:51:42 +02:00
Furquan Shaikh b0c2fe0554 cbfstool/fsp: Rename fsp1_1_relocate
FSP 2.0 uses the same relocate logic as FSP 1.1. Thus, rename
fsp1_1_relocate to more generic fsp_component_relocate that can be
used by cbfstool to relocate either FSP 1.1 or FSP 2.0
components. Allow FSP1.1 driver to still call fsp1_1_relocate which
acts as a wrapper for fsp_component_relocate.

Change-Id: I14a6efde4d86a340663422aff5ee82175362d1b0
Signed-off-by: Furquan Shaikh <furquan@google.com>
Reviewed-on: https://review.coreboot.org/14749
Tested-by: build bot (Jenkins)
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-by: Werner Zeh <werner.zeh@siemens.com>
2016-05-11 18:38:28 +02:00
Aaron Durbin 62c8dbe970 Revert "cbfstool: Add 'hashcbfs' command to compute hash of CBFS region."
This reverts commit 272a1f05b9.

In Chrome OS this command's usage was dropped in favor of another
solution. As it's not used drop the support for it.

Change-Id: I58b51446d3a8b5fed7fc391025225fbe38ffc007
Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: https://review.coreboot.org/14261
Tested-by: build bot (Jenkins)
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Martin Roth <martinroth@google.com>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2016-04-07 22:24:48 +02:00
Paul Menzel 8b831da586 Revert "cbfstool: Silence LZ4 -Wsign-compare warnings"
This reverts commit 17cb0370a7.

It’s the wrong thing to do, to just disable the warning. The code is
fixed for 32-bit user space now in Change-Id
I85bee25a69c432ef8bb934add7fd2e2e31f03662 (commonlib/lz4_wrapper: Use
correct casts to ensure valid calculations), so enable the warning
again.

Change-Id: I6d1c62c7b4875da8053c25e640c03cedf0ff2916
Signed-off-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-on: https://review.coreboot.org/13772
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Tested-by: build bot (Jenkins)
2016-02-24 19:27:13 +01:00
Julius Werner 17cb0370a7 cbfstool: Silence LZ4 -Wsign-compare warnings
It seems that the exact behavior of -Wsign-compare changes between GCC
versions... some of them like the commonlib/lz4_wrapper.c code, and some
don't. Since we don't have a well-defined HOSTCC toolchain this slipped
through pre-commit testing. Explicitly silence the warning to ensure
cbfstool still builds on all systems.

Change-Id: I43f951301d3f14ce34dadbe58e885b82d21d6353
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://review.coreboot.org/13769
Tested-by: build bot (Jenkins)
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-02-23 04:52:44 +01:00
Julius Werner 09f2921b5d cbfs: Add LZ4 in-place decompression support for pre-RAM stages
This patch ports the LZ4 decompression code that debuted in libpayload
last year to coreboot for use in CBFS stages (upgrading the base
algorithm to LZ4's dev branch to access the new in-place decompression
checks). This is especially useful for pre-RAM stages in constrained
SRAM-based systems, which previously could not be compressed due to
the size requirements of the LZMA scratchpad and bounce buffer. The
LZ4 algorithm offers a very lean decompressor function and in-place
decompression support to achieve roughly the same boot speed gains
(trading compression ratio for decompression time) with nearly no
memory overhead.

For now we only activate it for the stages that had previously not been
compressed at all on non-XIP (read: non-x86) boards. In the future we
may also consider replacing LZMA completely for certain boards, since
which algorithm wins out on boot speed depends on board-specific
parameters (architecture, processor speed, SPI transfer rate, etc.).

BRANCH=None
BUG=None
TEST=Built and booted Oak, Jerry, Nyan and Falco. Measured boot time on
Oak to be about ~20ms faster (cutting load times for affected stages
almost in half).

Change-Id: Iec256c0e6d585d1b69985461939884a54e3ab900
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://review.coreboot.org/13638
Tested-by: build bot (Jenkins)
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-02-22 21:38:37 +01:00
Aaron Durbin 272a1f05b9 cbfstool: Add 'hashcbfs' command to compute hash of CBFS region.
For the purposes of maintaining integrity of a CBFS allow one to
hash a CBFS over a given region. The hash consists of all file
metadata and non-empty file data. The resulting digest is saved
to the requested destination region.

BUG=chrome-os-partner:48412
BUG=chromium:445938
BRANCH=None
TEST=Integrated with glados chrome os build. vboot verification
     works using the same code to generate the hash in the tooling
     as well as at runtime on the board in question.

Change-Id: Ib0d6bf668ffd6618f5f73e1217bdef404074dbfc
Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: https://review.coreboot.org/12790
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2016-01-06 01:12:38 +01:00
Stefan Reinauer 354c11d10e cbfstool: Make sure fileno is available on Cygwin
This patch fixes compilation of cbfstool on Cygwin.

As reported in http://review.coreboot.org/#/c/10027
cbfstool on Cygwin likes to be compiled with -D_GNU_SOURCE.
That patch was abandoned because it would unwantedly turn on
more GNU extensions. Instead of doing that, only enable the
define on Cygwin, switch to -std=gnu99 instead of -std=c99 to
make fileno and strdup actually available.
A MINGW32 check that was forgotten in Makefile was copied over
from Makefile.inc to keep the two files in sync.

This patch has no impact on non-Windows builds.

Change-Id: I068b181d67daf9c7280110e64aefb634aa20c69b
Signed-off-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Reviewed-on: http://review.coreboot.org/11667
Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
Tested-by: build bot (Jenkins)
2015-10-23 20:44:35 +02:00
Aaron Durbin 8a414a0943 cbfstool: relocate FSP blobs on cbfstool add
When adding an FSP blob relocate it to its final
destination. This allows FSP to not be hard coded in
the cbfs. In order for the include paths to work
correctly w/ the edk 2 headers we need to supply
a neutered ProcessorBind.h to match up with the
tool environment such that one can get the UEFI
Platform Initialization type definitions.

BUG=chrome-os-partner:44827
BRANCH=None
TEST=Built glados and booted. Also added FSP with -b and manually
     adjusted location in fsp cache-as-ram. Booted as well.

Change-Id: I830d93578fdf745a51195109cf18d94a83ee8cd3
Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/11778
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-10-02 12:17:21 +00:00
Werner Zeh 8c3780a142 cbfstool: Allow anonymous structs/unions for older gcc versions
Older gcc versions throws a warning when a struct or union is
declared without a valid name (anonymous). This patch enables the
feature for older gcc versions so that no warning will be issued.

Change-Id: Idc5481f4d5723c5090a6f7d7dbb0686a737e11fc
Signed-off-by: Werner Zeh <werner.zeh@siemens.com>
Reviewed-on: http://review.coreboot.org/11779
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-10-02 08:54:51 +00:00
Patrick Georgi 89f20340d5 cbfstool: Add support for hashes as file metadata
They allow optimizing a verification of a whole CBFS image by only
dealing with the headers (assuming you choose to trust the hash
algorithm(s)).

The format allows for multiple hashes for a single file, and cbfstool
can handle them, but right now it can't generate such headers.

Loosely based on Sol's work in http://review.coreboot.org/#/c/10147/,
but using the compatible file attribute format. vboot is now a hard
dependency of the build process, but we import it into the tree for
quite a while now.

Change-Id: I9f14f30537d676ce209ad612e7327c6f4810b313
Signed-off-by: Patrick Georgi <patrick@georgi-clan.de>
Reviewed-on: http://review.coreboot.org/11767
Tested-by: build bot (Jenkins)
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-10-01 20:14:26 +00:00
Paul Kocialkowski 34ce438e6f cbfstool: Proper commonlib include path with no dependency on $(src)
$(src) is not defined when building directly from the cbfs directory (that is,
when building cbfs as standalone, running make in the cbfs directory), so we
need to define the path to the commonlib include path relative to $(top).

Change-Id: I72e80b030d4a156ec653ded5ab1457b16f612526
Signed-off-by: Paul Kocialkowski <contact@paulk.fr>
Reviewed-on: http://review.coreboot.org/11706
Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Tested-by: build bot (Jenkins)
2015-09-24 17:08:20 +00:00
Aaron Durbin dc9f5cd546 coreboot: introduce commonlib
Instead of reaching into src/include and re-writing code
allow for cleaner code sharing within coreboot and its
utilities. The additional thing needed at this point is
for the utilities to provide a printk() declaration within
a <console/console.h> file. That way code which uses printk()
can than be mapped properly to verbosity of utility parameters.

Change-Id: I9e46a279569733336bc0a018aed96bc924c07cdd
Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/11592
Tested-by: build bot (Jenkins)
Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
2015-09-22 21:21:34 +00:00
Patrick Georgi cdeb1c4f61 cbfstool: actually use no-ms-bitfields flag on mingw
It was added to an unused variable.

Change-Id: I869ffdda7e04b5c615931473c760d66b803fb98b
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Reviewed-on: http://review.coreboot.org/11673
Tested-by: build bot (Jenkins)
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-09-17 07:40:44 +00:00
Aaron Durbin 4be1674d32 cbfstool: add --xip support to add-stage for x86
Instead of going through the locate then add-stage
dance while linking romstage twice allow for adding romstage
with --xip flags to perform the relocation while adding it
into CBFS. The -P (page-size) and -a (alignment) parameters
were added as well so one could specify the necessary
parameters for x86 romstage.

BUG=chrome-os-partner:44827
BRANCH=None
TEST=Built and booted on glados.

Change-Id: I585619886f257e35f00961a1574009a51c28ff2b
Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/11669
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-09-16 14:11:10 +00:00
Jonathan A. Kollasch 9d6a17b549 util/cbfstool: use _XOPEN_SOURCE=700 to find strdup(3)
Change-Id: I974c6c8733356cc8ea4e0505136a34b6055abf0c
Signed-off-by: Jonathan A. Kollasch <jakllsch@kollasch.net>
Reviewed-on: http://review.coreboot.org/10809
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
Tested-by: build bot (Jenkins)
2015-07-07 02:14:08 +02:00
Patrick Georgi 7b32145763 cbfstool: Copy generated files to the build hierarchy before use
Instead of writing to the source tree (which we should generally avoid),
copy the pre-generated files (from lex and yacc) to $(objutil). Adapt
include paths and rules so they're found.

Change-Id: Id33be6d1dccf9a1b5857a29c55120dcc8f8db583
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Reviewed-on: http://review.coreboot.org/10252
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-05-19 20:30:11 +02:00
Kyösti Mälkki d32b27383b cbfstool: Rename autogenerated targets
These names will skip the lint-whitespace tests.

Change-Id: If4ac1f8e11fd0ac62f09696f2704477b6eb30046
Signed-off-by: Kyösti Mälkki <kyosti.malkki@gmail.com>
Reviewed-on: http://review.coreboot.org/10212
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-05-19 17:03:54 +02:00
Kyösti Mälkki c877149e9d cbfstool: Fix fmd_scanner build compatibility
Fixes these errors:
error: declaration of ‘input’ shadows a global declaration [-Werror=shadow]
error: redundant redeclaration of ‘isatty’ [-Werror=redundant-decls]

Change-Id: I4563d36e5389db4fdc5be3ca4e0e88af2642f7f8
Signed-off-by: Kyösti Mälkki <kyosti.malkki@gmail.com>
Reviewed-on: http://review.coreboot.org/10162
Tested-by: build bot (Jenkins)
Reviewed-by: David Hendricks <dhendrix@chromium.org>
Reviewed-by: Sol Boucher <solb@chromium.org>
Reviewed-by: Timothy Pearson <tpearson@raptorengineeringinc.com>
2015-05-19 07:17:36 +02:00
Sol Boucher 023e829423 fmaptool: Add listing of annotated CBFS sections and generate header
The fmd compiler now processes "(CBFS)" annotations, distilling them
into a comma-separated list of the names of sections containing
CBFSes. This list is the only thing printed to standard output to
enable easy capture and machine consumption by other tools.
Additionally, the ability to generate a tiny header with a define for
the primary CBFS's size is implemented and can be requested via a
new command-line switch.

Here's an example of how to use the new features:
 $ ./fmaptool -h layout.h layout_arm_8192.fmd layout.fmap 2>/dev/null
 FW_MAIN_A,FW_MAIN_B,COREBOOT
The hypothetical fmd file contains three sections annotated as (CBFS),
the names of which are printed to standard output. As before, a binary
FMAP file named layout.fmap is created; however, because the command
was invoked with -h, a header #define ing the offset of its FMAP
section (i.e. where it will be relative to the base of flash once the
boot image is assembled) is also generated.

BUG=chromium:470407
TEST=Verify that fmd files without a "COREBOOT" section or with one
that isn't annotated as "(CBFS)" are not accepted. Ensure that the
list of CBFS sections matches the descriptor file's annotations and
is led by the "COREBOOT" section. Invoke with the header generation
switch and check that output file for reasonableness.
BRANCH=None

Change-Id: I496dd937f69467bfd9233c28df59c7608e89538f
Signed-off-by: Sol Boucher <solb@chromium.org>
Original-Commit-Id: 9227698adecf675770b2983380eb570676c2b5d2
Original-Change-Id: I8b32f6ef19cabe2f6760106e676683c4565bbaad
Original-Signed-off-by: Sol Boucher <solb@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/262956
Original-Reviewed-by: Duncan Laurie <dlaurie@chromium.org>
Original-Reviewed-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: http://review.coreboot.org/9967
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-05-08 20:33:47 +02:00
Sol Boucher e3260a042f cbfstool: Restructure around support for reading/writing portions of files
The buffer API that cbfstool uses to read and write files only directly supports
one-shot operations on whole files. This adds an intermediate partitioned_file
module that sits on top of the buffer system and has an awareness of FMAP
entries. It provides an easy way to get a buffer for an individual region of a
larger image file based on FMAP section name, as well as incrementally write
those smaller buffers back to the backing file at the appropriate offset. The
module has two distinct modes of operation:
- For new images whose layout is described exclusively by an FMAP section, all
the aforementioned functionality will be available.
- For images in the current format, where the CBFS master header serves as the
root of knowledge of the image's size and layout, the module falls back to a
legacy operation mode, where it only allows manipulation of the entire image
as one unit, but exposes this support through the same interface by mapping
the region named SECTION_NAME_PRIMARY_CBFS ("COREBOOT") to the whole file.

The tool is presently only ported onto the new module running in legacy mode:
higher-level support for true "partitioned" images will be forthcoming. However,
as part of this change, the crusty cbfs_image_from_file() and
cbfs_image_write_file() abstractions are removed and replaced with a single
cbfs_image function, cbfs_image_from_buffer(), as well as centralized image
reading/writing directly in cbfstool's main() function. This reduces the
boilerplate required to implement each new action, makes the create action much
more similar to the others, and will make implementing additional actions and
adding in support for the new format much easier.

BUG=chromium:470407
TEST=Build panther and nyan_big coreboot.rom images with and without this patch
and diff their hexdumps. Ensure that no differences occur at different locations
from the diffs between subsequent builds of an identical source tree. Then flash
a full new build onto nyan_big and watch it boot normally.
BRANCH=None

Change-Id: I25578c7b223bc8434c3074cb0dd8894534f8c500
Signed-off-by: Sol Boucher <solb@chromium.org>
Original-Commit-Id: 7e1c96a48e7a27fc6b90289d35e6e169d5e7ad20
Original-Change-Id: Ia4a1a4c48df42b9ec2d6b9471b3a10eb7b24bb39
Original-Signed-off-by: Sol Boucher <solb@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/265581
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/10134
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-05-08 20:25:20 +02:00
Sol Boucher 69b88bf127 fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler
This adds a compiler for a language whose textual representation of flashmap
regions will be used to describe the layout of flash chips that contain more
than just a single CBFS. Direct integration with cbfstool (via a new
command-line switch for the create action) is forthcoming but will be added
separately.

BUG=chromium:461875
TEST=Use Chromium OS's cros_bundle_firmware script on the fmap.dts file for
panther. Using the latter file as a reference, write a corresponding
fmap.fmd file and feed it through fmaptool. Run both binary output files
though the flashmap project's own flashmap_decode utility. Observe only
the expected differences.
BRANCH=None

Change-Id: I06b32d138dbef0a4e5ed43c81bd31c796fd5d669
Signed-off-by: Sol Boucher <solb@chromium.org>
Original-Commit-Id: 005ab67eb594e21489cf31036aedaea87e0c7142
Original-Change-Id: Ia08f28688efdbbfc70c255916b8eb7eb0eb07fb2
Original-Signed-off-by: Sol Boucher <solb@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/255031
Original-Reviewed-by: Julius Werner <jwerner@chromium.org>
Original-Reviewed-by: Stefan Reinauer <reinauer@chromium.org>
Reviewed-on: http://review.coreboot.org/9942
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-05-08 19:55:42 +02:00
Paul Menzel 6548ae9f99 cbfstool/Makefile*: Use `LDFLAGS` instead of `LINKFLAGS`
Commit 0e53931f (cbfstool: Clean up in preparation for adding new
files) split out the flags and introduced the variable `LINKFLAGS`.
Rename it to `LDFLAGS` which is more commonly used.

Change-Id: Ib6299f8ef5cf30dbe05bfae36f30ae4371f0a738
Signed-off-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-on: http://review.coreboot.org/10064
Tested-by: build bot (Jenkins)
Reviewed-by: Sol Boucher <solb@chromium.org>
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-05-04 12:09:59 +02:00
Sol Boucher 0e53931fee cbfstool: Clean up in preparation for adding new files
This enables more warnings on the cbfstool codebase and fixes the
issues that surface as a result. A memory leak that used to occur
when compressing files with lzma is also found and fixed.
Finally, there are several fixes for the Makefile:
 - Its autodependencies used to be broken because the target for
   the .dependencies file was misnamed; this meant that Make
   didn't know how to rebuild the file, and so would silently
   skip the step of updating it before including it.
 - The ability to build to a custom output directory by defining
   the obj variable had bitrotted.
 - The default value of the obj variable was causing implicit
   rules not to apply when specifying a file as a target without
   providing a custom value for obj.
 - Add a distclean target for removing the .dependencies file.

BUG=chromium:461875
TEST=Build an image with cbfstool both before and after.
BRANCH=None

Change-Id: I951919d63443f2b053c2e67c1ac9872abc0a43ca
Signed-off-by: Sol Boucher <solb@chromium.org>
Original-Commit-Id: 49293443b4e565ca48d284e9a66f80c9c213975d
Original-Change-Id: Ia7350c2c3306905984cfa711d5fc4631f0b43d5b
Original-Signed-off-by: Sol Boucher <solb@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/257340
Original-Reviewed-by: Julius Werner <jwerner@chromium.org>
Original-Reviewed-by: Stefan Reinauer <reinauer@chromium.org>
Reviewed-on: http://review.coreboot.org/9937
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-04-25 12:14:25 +02:00
Patrick Georgi 913e1718cd build system: fix another cbfstool race
It just doesn't work to have files depend on their parent
directory: As soon as the files are written, the time stamp
of the directory changes, too.

This led to spurious updates of cbfstool and rmodtool, and
related "permission denied" errors when linker and build
system ran into each other.

Change-Id: I44a7d7b4b1d47a1567ece1f57dfd6745d05ee651
Signed-off-by: Patrick Georgi <patrick@georgi-clan.de>
Reviewed-on: http://review.coreboot.org/6276
Tested-by: build bot (Jenkins)
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Kyösti Mälkki <kyosti.malkki@gmail.com>
2014-07-15 14:27:04 +02:00
Edward O'Callaghan 44fd0a0031 utils/cbfstool: No need to pass -g flag twice
Spotted by building with Clang.

Change-Id: I7ab97278d8bd586a71e453c8cc9d26dd6938c8d2
Signed-off-by: Edward O'Callaghan <eocallaghan@alterapraxis.com>
Reviewed-on: http://review.coreboot.org/6139
Tested-by: build bot (Jenkins)
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Idwer Vollering <vidwer@gmail.com>
2014-06-29 04:16:49 +02:00
Aaron Durbin 4fde5a66b4 util: add rmodtool for parsing ELF files to rmodules
The current implementation of creating rmodules relies
on invoking the linker in a certain manner with the
relocations overlaid on the BSS section. It's not really
surprising that the linker doesn't always behave the way
one wants depending on the linker used and the architecture.
Instead, introduce rmodtool which takes an ELF file as an
input, parses it, and creates a new ELF file in the format
the rmodule loader expects.

Change-Id: I31ac2d327d450ef841c3a7d9740b787278382bef
Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/5378
Tested-by: build bot (Jenkins)
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
2014-03-20 21:34:39 +01:00
Alexandru Gagniuc 5edfa3779d cbfstool/lzma: Remove dead code under #ifdefs
Remove a bunch of dead code which depends either on commented out
#defines, or compiler definitions. Use this opportunity to remove the
need for "-D_7ZIP_ST" in the compiler flags.

Change-Id: Ib6629002be7bf4cee6d95d7baa724893b5e8ba32
Signed-off-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Reviewed-on: http://review.coreboot.org/5083
Tested-by: build bot (Jenkins)
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
Reviewed-by: Edward O'Callaghan <eocallaghan@alterapraxis.com>
2014-03-02 05:14:30 +01:00
Ronald G. Minnich a8a133ded3 Add section header parsing and use it in the mk-payload step
This completes the improvements to the ELF file parsing code.  We can
now parse section headers too, across all 4 combinations of word size
and endianness. I had hoped to completely remove the use of htonl
until I found it in cbfs_image.c. That's a battle for another day.

There's now a handy macro to create magic numbers in host byte order.
I'm using it for all the PAYLOAD_SEGMENT_* constants and maybe
we can use it for the others too, but this is sensitive code and
I'd rather change one thing at a time.

To maximize the ease of use for users, elf parsing is accomplished with
just one function:

int
elf_headers(const struct buffer *pinput,
	    Elf64_Ehdr *ehdr,
	    Elf64_Phdr **pphdr,
	    Elf64_Shdr **pshdr)

which requires the ehdr and pphdr pointers to be non-NULL, but allows
the pshdr to be NULL. If pshdr is NULL, the code will not try to read
in section headers.

To satisfy our powerful scripts, I had to remove the ^M from an unrelated
microcode file.

BUG=None
TEST=Build a peppy image (known to boot) with old and new versions and verify they are bit-for-bit the same. This was also fully tested across all chromebooks for building and booting and running chromeos.
BRANCH=None

Change-Id: I54dad887d922428b6175fdb6a9cdfadd8a6bb889
Signed-off-by: Ronald G. Minnich <rminnich@google.com>
Reviewed-on: https://chromium-review.googlesource.com/181272
Reviewed-by: Ronald Minnich <rminnich@chromium.org>
Commit-Queue: Ronald Minnich <rminnich@chromium.org>
Tested-by: Ronald Minnich <rminnich@chromium.org>
Signed-off-by: Ronald G. Minnich <rminnich@google.com>
Reviewed-on: http://review.coreboot.org/5098
Tested-by: build bot (Jenkins)
Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
2014-02-02 20:18:55 +01:00
Ronald G. Minnich aa2f739ae8 cbfs: fix issues with word size and endianness.
Add XDR functions and use them to convert the ELF headers
to native headers, using the Elf64 structs to ensure we accomodate
all word sizes. Also, use these XDR functions for output.

This may seem overly complex but it turned out to be much the easiest
way to do this. Note that the basic elf parsing function
in cbfs-mkstage.c now works over all ELF files, for all architectures,
endian, and word size combinations. At the same time, the basic elf
parsing in cbfs-mkstage.c is a loop that has no architecture-specific
conditionals.

Add -g to the LDFLAGS while we're here. It's on the CFLAGS so there is
no harm done.

This code has been tested on all chromebooks that use coreboot to date.

I added most of the extra checks from ChromeOS and they triggered a
lot of warnings, hence the other changes. I had to take -Wshadow back
out due to the many errors it triggers in LZMA.

BUG=None
TEST=Build and boot for Peppy; works fine. Build and boot for nyan,
     works fine. Build for qemu targets and armv8 targets.
BRANCH=None

Change-Id: I5a4cee9854799189115ac701e22efc406a8d902f
Signed-off-by: Ronald G. Minnich <rminnich@google.com>
Reviewed-on: https://chromium-review.googlesource.com/178606
Reviewed-by: Ronald Minnich <rminnich@chromium.org>
Commit-Queue: Ronald Minnich <rminnich@chromium.org>
Tested-by: Ronald Minnich <rminnich@chromium.org>
Reviewed-on: http://review.coreboot.org/4817
Reviewed-by: Alexandru Gagniuc <mr.nuke.me@gmail.com>
Tested-by: build bot (Jenkins)
Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
2014-01-29 20:03:44 +01:00
Patrick Georgi de36d333c2 Add a (b)zImage parser to cbfstool
In the great tradition of LinuxBIOS this allows adding
a kernel as payload. add-payload is extended to also
allow adding an initial ramdisk (-I filename) and a
command line (-C console=ttyS0).

Change-Id: Iaca499a98b0adf0134e78d6bf020b6531a626aaa
Signed-off-by: Patrick Georgi <patrick.georgi@secunet.com>
Signed-off-by: Patrick Georgi <patrick@georgi-clan.de>
Reviewed-on: http://review.coreboot.org/3302
Tested-by: build bot (Jenkins)
Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
2013-08-31 08:58:34 +02:00
Stefan Reinauer aa3f7ba36e cbfstool: Replace C++ code with C code
cbfstool was using a C++ wrapper around the C written LZMA functions.
And a C wrapper around those C++ functions. Drop the mess and rewrite
the functions to be all C.

Change-Id: Ieb6645a42f19efcc857be323ed8bdfcd9f48ee7c
Signed-off-by: Stefan Reinauer <reinauer@google.com>
Reviewed-on: http://review.coreboot.org/3010
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Tested-by: build bot (Jenkins)
Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
2013-04-03 02:35:28 +02:00
Aaron Durbin 6b0d0d6e14 cbfstool: Add update-fit command
Add support for filling in the Firmware Interface Table.
For now it only supports adding microcode entries.

It takes 2 options:
1. Name of file in cbfs where the mircocode is located
2. The number of empty entries in the table.

Verified with go firmware tools. Also commented out updating
microcode in the bootblock. When romstage runs, the CPUs indicate
their microcode is already loaded.

Change-Id: Iaccaa9c226ee24868a5f4c0ba79729015d15bbef
Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Signed-off-by: Stefan Reinauer <reinauer@google.com>
Reviewed-on: http://review.coreboot.org/2712
Reviewed-by: Aaron Durbin <adurbin@google.com>
Tested-by: build bot (Jenkins)
2013-03-27 01:25:12 +01:00
Hung-Te Lin eab2c81949 cbfstool: Add cbfs_image new CBFS image manipulation API.
Current cbfstool implementation is relying on global variables to pass processed
data, and the calculation of address is based on x86 architecture (ex, always
assuming 0x0000 as invalid address), not easy to be used on platforms without
top-aligned memory mapping. This CL is a first step to start a new cbfstool
without global variables, and to prevent assuming memory layout in x86 mode.

The first published APIs are for reading and writing existing CBFS ROM image
files (and to find file entries in a ROM file).

Read cbfs_image.h for detail usage of each API function.

Change-Id: I28c737c8f290e51332119188248ac9e28042024c
Signed-off-by: Hung-Te Lin <hungte@chromium.org>
Reviewed-on: http://review.coreboot.org/2194
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2013-02-05 22:25:02 +01:00
Zheng Bao a182cbdd62 cbfstool: Align the column of build hint message.
Change-Id: Ic217450411d7fa4e6c3a053be62d7c948dc7145e
Signed-off-by: Zheng Bao <zheng.bao@amd.com>
Signed-off-by: zbao <fishbaozi@gmail.com>
Reviewed-on: http://review.coreboot.org/2030
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <patrick@georgi-clan.de>
2012-12-14 08:46:26 +01:00
Stefan Reinauer 2e200cde9a cbfstool: Update LZMA encoder to LZMA SDK 9.12
This removes almost all C++ code (except the wrapper)

Change-Id: I0f84070e3b6dc57c98d49a53150a140479b3221f
Signed-off-by: Stefan Reinauer <reinauer@google.com>
Reviewed-on: http://review.coreboot.org/1799
Tested-by: build bot (Jenkins)
Reviewed-by: Ronald G. Minnich <rminnich@gmail.com>
2012-11-12 18:35:52 +01:00
Zheng Bao c31cdd8662 cbfstool: Add -mno-ms-bitfields on (mingw)
The default gcc on mingw will process the __attribute__ ((packed)) in
a different way other than non-win system.

Change-Id: Iac9f4476c922472d0b447f1c3ef60e8e13bd902f
Signed-off-by: Zheng Bao <zheng.bao@amd.com>
Signed-off-by: Zheng Bao <fishbaozi@gmail.com>
Reviewed-on: http://review.coreboot.org/1603
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <patrick@georgi-clan.de>
2012-10-22 21:49:46 +02:00
Patrick Georgi a2acbc70a3 Build utils into their source directory equivalent in
the build tree.
Allow separate build tree for utils
Use separate build tree for utils in abuild

Signed-off-by: Patrick Georgi <patrick.georgi@coresystems.de>
Acked-by: Stefan Reinauer <stepan@coresystems.de>


git-svn-id: svn://svn.coreboot.org/coreboot/trunk@5453 2b7e53f0-3cfb-0310-b3e9-8179ed1497e1
2010-04-16 22:48:57 +00:00
Patrick Georgi 51e142fef4 make only needs to read Makefile.incs once, thanks to the
SECONDEXPANSION feature of GNU make (and we rely on GNU make for lots of
things already)

File paths are relative to the root directory, which simplifies
debugging (make V=1 gives shorter command lines) and helps ccache
finding matches for checkouts in different directories (even though it
should normalize paths itself)

Signed-off-by: Patrick Georgi <patrick.georgi@coresystems.de>
Acked-by: Ronald G. Minnich <rminnich@gmail.com>


git-svn-id: svn://svn.coreboot.org/coreboot/trunk@5304 2b7e53f0-3cfb-0310-b3e9-8179ed1497e1
2010-03-27 17:18:39 +00:00
Uwe Hermann 6c73b4416c Remove all build/ prefixes in the build output.
Also, remove one missing hardcoded "build" dir in the distclean target,
and clean up files generated by sconfig in 'make clean'.

Signed-off-by: Uwe Hermann <uwe@hermann-uwe.de>
Acked-by: Myles Watson <mylesgw@gmail.com>



git-svn-id: svn://svn.coreboot.org/coreboot/trunk@4880 2b7e53f0-3cfb-0310-b3e9-8179ed1497e1
2009-10-28 17:10:51 +00:00
Uwe Hermann 7782319c21 Prefix all build output file names of files which end up in the build
directory with "build/" for consistency (trivial, sort of).

Also, drop printing of "config.g" input file, we usually only print
generated/output files in the build output.

Finally, rename non-existing COMPRESSFLAG variable to
CBFS_PAYLOAD_COMPRESS_FLAG in a printf line. The build output now says

    PAYLOAD    payload.elf l

for payloads (the "l" specifies LZMA compression).

Signed-off-by: Uwe Hermann <uwe@hermann-uwe.de>
Acked-by: Uwe Hermann <uwe@hermann-uwe.de>



git-svn-id: svn://svn.coreboot.org/coreboot/trunk@4875 2b7e53f0-3cfb-0310-b3e9-8179ed1497e1
2009-10-27 23:14:54 +00:00
Patrick Georgi ab13458c34 Build cbfstool with debug flags and don't strip the executable
from any symbol information.

Variant of Ron's patch.

Signed-off-by: Patrick Georgi <patrick.georgi@coresystems.de>
Acked-by: Patrick Georgi <patrick.georgi@coresystems.de>


git-svn-id: svn://svn.coreboot.org/coreboot/trunk@4637 2b7e53f0-3cfb-0310-b3e9-8179ed1497e1
2009-09-16 08:19:07 +00:00
Patrick Georgi 77ee932a22 Build cbfstool with native data size. Trivial change.
Signed-off-by: Patrick Georgi <patrick.georgi@coresystems.de>
Acked-by: Patrick Georgi <patrick.georgi@coresystems.de>


git-svn-id: svn://svn.coreboot.org/coreboot/trunk@4631 2b7e53f0-3cfb-0310-b3e9-8179ed1497e1
2009-09-14 14:13:13 +00:00
Patrick Georgi b7b56dd8fb New cbfstool. Works without mmap or fork/exec and
supports fixed location files. Some parts are salvaged
from the pre-commit version (esp. stage and payload creation),
others are completely rewritten (eg. the main loop that handles
file addition)

Also adapt newconfig (we don't need cbfs/tools anymore) and fix
some minor issues in the cbfstool-README.

Signed-off-by: Patrick Georgi <patrick.georgi@coresystems.de>
Acked-by: Stefan Reinauer <stepan@coresystems.de>


git-svn-id: svn://svn.coreboot.org/coreboot/trunk@4630 2b7e53f0-3cfb-0310-b3e9-8179ed1497e1
2009-09-14 13:29:27 +00:00
Patrick Georgi 2b7418ecc6 Use the gnu make .SILENT: mechanism instead of requiring
$(Q) in front of every silent line.

make V=1 or make Q="" still make make noisy again.

Signed-off-by: Patrick Georgi <patrick.georgi@coresystems.de>
Acked-by: Ronald G. Minnich <rminnich@gmail.com>


git-svn-id: svn://svn.coreboot.org/coreboot/trunk@4581 2b7e53f0-3cfb-0310-b3e9-8179ed1497e1
2009-08-25 19:38:46 +00:00
Uwe Hermann 5ec2c2b998 Various Kconfig and Makefile.inc fixes and cosmetics.
- Whitespace fixes, remove trailing whitespace, use TABs for identation
   (except in Kconfig "help" lines, which start with one TAB and two spaces
   as per Linux kernel style)

 - Kconfig: Standardize on 'bool' (not 'boolean').

 - s/lar/cbfs/ in one Kconfig help string.

 - Reword various Kconfig menu entries for a more usable and consistent menu.

 - Fix incorrect comment of NO_RUN in devices/Kconfig.

 - superio/serverengines/Kconfig: Incorrect config name.

 - superio/Makefile.inc: s/serverengine/serverengines/.

 - superio/intel/Kconfig: s/SUPERIO_FINTEK_I3100/SUPERIO_INTEL_I3100/.

 - mainboard/via/vt8454c/Kconfig: Fix copy-paste error in help string.

 - mainboard/via/epia-n/Kconfig: Fix "bool" menu text.

 - console/Kconfig: Don't mention defaults in the menu string, kconfig
   already displays them anyway.

 - Kill "Drivers" menu for now, it only confuses users as long as it's emtpy.

Signed-off-by: Uwe Hermann <uwe@hermann-uwe.de>
Acked-by: Uwe Hermann <uwe@hermann-uwe.de>



git-svn-id: svn://svn.coreboot.org/coreboot/trunk@4567 2b7e53f0-3cfb-0310-b3e9-8179ed1497e1
2009-08-25 00:53:22 +00:00