coreboot-kgpe-d16/util/cbfstool
Julius Werner 09f2921b5d cbfs: Add LZ4 in-place decompression support for pre-RAM stages
This patch ports the LZ4 decompression code that debuted in libpayload
last year to coreboot for use in CBFS stages (upgrading the base
algorithm to LZ4's dev branch to access the new in-place decompression
checks). This is especially useful for pre-RAM stages in constrained
SRAM-based systems, which previously could not be compressed due to
the size requirements of the LZMA scratchpad and bounce buffer. The
LZ4 algorithm offers a very lean decompressor function and in-place
decompression support to achieve roughly the same boot speed gains
(trading compression ratio for decompression time) with nearly no
memory overhead.

For now we only activate it for the stages that had previously not been
compressed at all on non-XIP (read: non-x86) boards. In the future we
may also consider replacing LZMA completely for certain boards, since
which algorithm wins out on boot speed depends on board-specific
parameters (architecture, processor speed, SPI transfer rate, etc.).

BRANCH=None
BUG=None
TEST=Built and booted Oak, Jerry, Nyan and Falco. Measured boot time on
Oak to be about ~20ms faster (cutting load times for affected stages
almost in half).

Change-Id: Iec256c0e6d585d1b69985461939884a54e3ab900
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://review.coreboot.org/13638
Tested-by: build bot (Jenkins)
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-02-22 21:38:37 +01:00
..
console tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
flashmap cbfstool: Change FMAP granularity to 16 bytes 2016-01-14 18:44:41 +01:00
lz4 cbfs: Add LZ4 in-place decompression support for pre-RAM stages 2016-02-22 21:38:37 +01:00
lzma cbfstool: have decompress functions provide ouput data size 2015-10-25 13:23:15 +01:00
EXAMPLE cbfstool: Update example file. 2013-02-04 11:12:15 +01:00
Makefile cbfstool: remove trampoline_start and trampoline_size 2015-12-02 18:43:23 +01:00
Makefile.inc cbfs: Add LZ4 in-place decompression support for pre-RAM stages 2016-02-22 21:38:37 +01:00
ProcessorBind.h cbfstool: relocate FSP blobs on cbfstool add 2015-10-02 12:17:21 +00:00
README.fmaptool fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
cbfs-mkpayload.c tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
cbfs-mkstage.c cbfs: Add LZ4 in-place decompression support for pre-RAM stages 2016-02-22 21:38:37 +01:00
cbfs-payload-linux.c cbfstool: Fix checkpatch error 2015-12-02 18:43:47 +01:00
cbfs.h cbfstool: Add attributes for position and alignment constraints. 2016-01-22 06:26:15 +01:00
cbfs_hash.c cbfstool: Add 'hashcbfs' command to compute hash of CBFS region. 2016-01-06 01:12:38 +01:00
cbfs_image.c cbfs: Add LZ4 in-place decompression support for pre-RAM stages 2016-02-22 21:38:37 +01:00
cbfs_image.h util/cbfstool: add 'compact' command 2016-01-28 19:25:57 +01:00
cbfs_sections.c tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
cbfs_sections.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
cbfstool.c util/cbfstool: add 'compact' command 2016-01-28 19:25:57 +01:00
coff.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
common.c cbfstool: add ppc64 support 2015-12-12 01:36:51 +01:00
common.h cbfs: Add LZ4 in-place decompression support for pre-RAM stages 2016-02-22 21:38:37 +01:00
compress.c cbfs: Add LZ4 in-place decompression support for pre-RAM stages 2016-02-22 21:38:37 +01:00
default-x86.fmd build system: Switch to fmap based firmware layout 2015-12-11 11:40:18 +01:00
default.fmd build system: Switch to fmap based firmware layout 2015-12-11 11:40:18 +01:00
elf.h cbfstool: add ppc64 support 2015-12-12 01:36:51 +01:00
elfheaders.c cbfstool: add ppc64 support 2015-12-12 01:36:51 +01:00
elfparsing.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
fit.c tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
fit.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
flashmap_tests.c tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
fmap_from_fmd.c tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
fmap_from_fmd.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
fmaptool.c cbfstool: Fix compile issue for older gcc versions 2016-01-26 06:48:15 +01:00
fmd.c tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
fmd.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
fmd_parser.c_shipped cbfstool: Rename autogenerated targets 2015-05-19 17:03:54 +02:00
fmd_parser.h_shipped cbfstool: Rename autogenerated targets 2015-05-19 17:03:54 +02:00
fmd_parser.y tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
fmd_scanner.c_shipped Remove empty lines at end of file 2015-06-08 00:55:07 +02:00
fmd_scanner.h_shipped cbfstool: Rename autogenerated targets 2015-05-19 17:03:54 +02:00
fmd_scanner.l tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
fv.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
linux.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
linux_trampoline.S cbfstool: autocreate trampoline 2015-12-02 18:43:01 +01:00
linux_trampoline.c cbfstool: remove trampoline_start and trampoline_size 2015-12-02 18:43:23 +01:00
linux_trampoline.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
option.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
partitioned_file.c cbfstool: accept read-only files when possible 2016-01-19 18:14:44 +01:00
partitioned_file.h cbfstool: accept read-only files when possible 2016-01-19 18:14:44 +01:00
rmodtool.c tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
rmodule.c tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
rmodule.h tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00
swab.h cbfstool: Add header file for ntohl & htonl on Apple 2016-01-20 16:10:20 +01:00
xdr.c tree: drop last paragraph of GPL copyright header 2015-10-31 21:37:39 +01:00

README.fmaptool

Flashmap descriptors in coreboot
================================
Flashmap (https://code.google.com/p/flashmap) is a binary format for representing the layout of
flash chips. Since coreboot is starting to use a "partition" of this format to describe the flash
chip layout---both at runtime and when flashing a new image onto a chip---, the project needed a
reasonably expressive plaintext format for representing such sections in the source tree. Our
solution is the fmd ("flashmap descriptor") language, and the files in this directory contain a
scanner, parser, semantic analyser, and flashmap converter. Here's an informal language description:

# <line comment>
<image name>[@<memory-mapped address>] <image size> {
	<section name>[@<offset from start of image>] [<section size>] [{
		<subsection name>[@<offset from start of parent section>] [<subsection size>] [{
			# Sections can be nested as deeply as desired
			<subsubsection name>[(CBFS)][@...] [...] [{...}]
		}]
		[<subsection name>[(CBFS)][@...] [...] [{...}]]
		# There can be many subsections at each level of nesting: they will be inserted
		# sequentially, and although gaps are allowed, any provided offsets are always
		# relative to the closest parent node's and must be strictly increasing with neither
		# overlapping nor degenerate-size sections.
	}]
}

Note that the above example contains a few symbols that are actually metasyntax, and therefore have
neither meaning nor place in a real file. The <.*> s indicate placeholders for parameters:
 - The names are strings, which are provided as single-word---no whitespace---groups of
   syntactically unimportant symbols (i.e. everything except @, {, and }): they are not surrounded
   by quotes or any other form of delimiter.
 - The other fields are nonnegative integers, which may be given as decimal or hexadecimal; in
   either case, a K, M, or G may be appended---without intermediate whitespace---as a multiplier.
 - Comments consist of anything one manages to enter, provided it doesn't start a new line.
The [.*] s indicate that a portion of the file could be omitted altogether:
 - Just because something is noted as optional doesn't mean it is in every case: the answer might
   actually depend on which other information is---or isn't---provided.
 - In particular, it is only legal to place a (CBFS) annotation on a leaf section; that is, choosing
   to add child sections excludes the possibility of putting a CBFS in their parent. Such
   annotations are only used to decide where CBFS empty file headers should be created, and do not
   result in the storage of any additional metadata in the resulting FMAP section.
Additionally, it's important to note these properties of the overall file and its values:
 - Other than within would-be strings and numbers, whitespace is ignored. It goes without saying
   that such power comes with responsibility, which is why this sentence is here.
 - Although the .*section names must be globally unique, one of them may---but is not required to---
   match the image name.
 - It is a syntax error to supply a number---besides 0---that begins with the character 0, as there
   is no intention of adding octals to the mix.
 - The image's memory address should be present on---and only on---layouts for memory-mapped chips.
 - Although it may be evident from above, all .*section offsets are relative only to the immediate
   parent. There is no way to include an absolute offset (i.e. from the beginning of flash), which
   means that it is "safe" to reorder the .*section s within a particular level of nesting, as long
   as the change doesn't cause their positions and sizes to necessitate overlap or zero sizes.
 - A .*section with omitted offset is assumed to start at as low a position as possible---with no
   consideration of alignment---and one with omitted size is assumed to fill the remaining space
   until the next sibling or before the end of its parent.
 - It's fine to omit any .*section 's offset, size, or both, provided its position and size are
   still unambiguous in the context of its *sibling* sections and its parent's *size*. In
   particular, knowledge of one .*section 's children or the .*section s' common parent's siblings
   will not be used for this purpose.
 - Although .*section s are not required to have children, the flash chip as a whole must have at
   least one.
 - Though the braces after .*section s may be omitted for those that have no children, if they are
   present, they must contain at least one child.

PL people and sympathizers may wish to examine the formal abstract syntax and context-free grammar,
which are located in fmd_scanner.l and fmd_scanner.y, respectively. Those interested in the
algorithm used to infer omitted values will feel at home in fmd.c, particularly near the definition
of validate_and_complete_info().