coreboot-kgpe-d16/util/cbfstool
Sol Boucher e3260a042f cbfstool: Restructure around support for reading/writing portions of files
The buffer API that cbfstool uses to read and write files only directly supports
one-shot operations on whole files. This adds an intermediate partitioned_file
module that sits on top of the buffer system and has an awareness of FMAP
entries. It provides an easy way to get a buffer for an individual region of a
larger image file based on FMAP section name, as well as incrementally write
those smaller buffers back to the backing file at the appropriate offset. The
module has two distinct modes of operation:
- For new images whose layout is described exclusively by an FMAP section, all
the aforementioned functionality will be available.
- For images in the current format, where the CBFS master header serves as the
root of knowledge of the image's size and layout, the module falls back to a
legacy operation mode, where it only allows manipulation of the entire image
as one unit, but exposes this support through the same interface by mapping
the region named SECTION_NAME_PRIMARY_CBFS ("COREBOOT") to the whole file.

The tool is presently only ported onto the new module running in legacy mode:
higher-level support for true "partitioned" images will be forthcoming. However,
as part of this change, the crusty cbfs_image_from_file() and
cbfs_image_write_file() abstractions are removed and replaced with a single
cbfs_image function, cbfs_image_from_buffer(), as well as centralized image
reading/writing directly in cbfstool's main() function. This reduces the
boilerplate required to implement each new action, makes the create action much
more similar to the others, and will make implementing additional actions and
adding in support for the new format much easier.

BUG=chromium:470407
TEST=Build panther and nyan_big coreboot.rom images with and without this patch
and diff their hexdumps. Ensure that no differences occur at different locations
from the diffs between subsequent builds of an identical source tree. Then flash
a full new build onto nyan_big and watch it boot normally.
BRANCH=None

Change-Id: I25578c7b223bc8434c3074cb0dd8894534f8c500
Signed-off-by: Sol Boucher <solb@chromium.org>
Original-Commit-Id: 7e1c96a48e7a27fc6b90289d35e6e169d5e7ad20
Original-Change-Id: Ia4a1a4c48df42b9ec2d6b9471b3a10eb7b24bb39
Original-Signed-off-by: Sol Boucher <solb@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/265581
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/10134
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-05-08 20:25:20 +02:00
..
flashmap cbfstool: Restructure around support for reading/writing portions of files 2015-05-08 20:25:20 +02:00
lzma cbfstool: Clean up in preparation for adding new files 2015-04-25 12:14:25 +02:00
cbfs-mkpayload.c cbfstool: Clean up in preparation for adding new files 2015-04-25 12:14:25 +02:00
cbfs-mkstage.c cbfstool: Clean up in preparation for adding new files 2015-04-25 12:14:25 +02:00
cbfs-payload-linux.c cbfstool:linux_trampoline: config CS and DS segment descriptors 2014-09-04 23:34:32 +02:00
cbfs.h cbfstool: add filetypes for FSP, MRC, SPC and MRC_CACHE 2015-03-30 21:45:11 +02:00
cbfs_image.c cbfstool: Restructure around support for reading/writing portions of files 2015-05-08 20:25:20 +02:00
cbfs_image.h cbfstool: Restructure around support for reading/writing portions of files 2015-05-08 20:25:20 +02:00
cbfstool.c cbfstool: Restructure around support for reading/writing portions of files 2015-05-08 20:25:20 +02:00
coff.h GPLv2 notice: Unify all files to just use one space in »MA 02110-1301« 2013-03-01 10:16:08 +01:00
common.c cbfstool: Restructure around support for reading/writing portions of files 2015-05-08 20:25:20 +02:00
common.h cbfstool: Restructure around support for reading/writing portions of files 2015-05-08 20:25:20 +02:00
compress.c cbfstool: Propogate compression errors back to the caller. 2014-09-25 20:26:04 +02:00
elf.h cbfstool: Add relocation codes for arm mode 2015-03-17 16:53:50 +01:00
elfheaders.c cbfstool: Clean up in preparation for adding new files 2015-04-25 12:14:25 +02:00
elfparsing.h cbfstool: Clean up in preparation for adding new files 2015-04-25 12:14:25 +02:00
EXAMPLE cbfstool: Update example file. 2013-02-04 11:12:15 +01:00
fit.c cbfstool: Clean up in preparation for adding new files 2015-04-25 12:14:25 +02:00
fit.h cbfstool: Add update-fit command 2013-03-27 01:25:12 +01:00
flashmap_tests.c fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmap_from_fmd.c fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmap_from_fmd.h fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmaptool.c fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmd.c fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmd.h fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmd_parser.c fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmd_parser.h fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmd_parser.y fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmd_scanner.c fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmd_scanner.h fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fmd_scanner.l fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
fv.h GPLv2 notice: Unify all files to just use one space in »MA 02110-1301« 2013-03-01 10:16:08 +01:00
linux.h cbfstool:linux_trampoline: config CS and DS segment descriptors 2014-09-04 23:34:32 +02:00
linux_trampoline.c cbfstool:linux_trampoline: config CS and DS segment descriptors 2014-09-04 23:34:32 +02:00
linux_trampoline.h cbfstool:linux_trampoline: config CS and DS segment descriptors 2014-09-04 23:34:32 +02:00
Makefile cbfstool: Restructure around support for reading/writing portions of files 2015-05-08 20:25:20 +02:00
Makefile.inc cbfstool: Restructure around support for reading/writing portions of files 2015-05-08 20:25:20 +02:00
option.h fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
partitioned_file.c cbfstool: Restructure around support for reading/writing portions of files 2015-05-08 20:25:20 +02:00
partitioned_file.h cbfstool: Restructure around support for reading/writing portions of files 2015-05-08 20:25:20 +02:00
README.fmaptool fmaptool: Introduce the fmd ("flashmap descriptor") language and compiler 2015-05-08 19:55:42 +02:00
rmodtool.c util: add rmodtool for parsing ELF files to rmodules 2014-03-20 21:34:39 +01:00
rmodule.c cbfstool: Clean up in preparation for adding new files 2015-04-25 12:14:25 +02:00
rmodule.h util: add rmodtool for parsing ELF files to rmodules 2014-03-20 21:34:39 +01:00
swab.h Various fixes to cbfstool. 2011-10-24 20:29:29 +02:00
xdr.c cbfstool: add bputs() to store a byte stream to a buffer 2014-03-11 19:43:17 +01:00

Flashmap descriptors in coreboot
================================
Flashmap (https://code.google.com/p/flashmap) is a binary format for representing the layout of
flash chips. Since coreboot is starting to use a "partition" of this format to describe the flash
chip layout---both at runtime and when flashing a new image onto a chip---, the project needed a
reasonably expressive plaintext format for representing such sections in the source tree. Our
solution is the fmd ("flashmap descriptor") language, and the files in this directory contain a
scanner, parser, semantic analyser, and flashmap converter. Here's an informal language description:

# <line comment>
<image name>[@<memory-mapped address>] <image size> {
	<section name>[@<offset from start of image>] [<section size>] [{
		<subsection name>[@<offset from start of parent section>] [<subsection size>] [{
			# Sections can be nested as deeply as desired
			<subsubsection name>[(CBFS)][@...] [...] [{...}]
		}]
		[<subsection name>[(CBFS)][@...] [...] [{...}]]
		# There can be many subsections at each level of nesting: they will be inserted
		# sequentially, and although gaps are allowed, any provided offsets are always
		# relative to the closest parent node's and must be strictly increasing with neither
		# overlapping nor degenerate-size sections.
	}]
}

Note that the above example contains a few symbols that are actually metasyntax, and therefore have
neither meaning nor place in a real file. The <.*> s indicate placeholders for parameters:
 - The names are strings, which are provided as single-word---no whitespace---groups of
   syntactically unimportant symbols (i.e. everything except @, {, and }): they are not surrounded
   by quotes or any other form of delimiter.
 - The other fields are nonnegative integers, which may be given as decimal or hexadecimal; in
   either case, a K, M, or G may be appended---without intermediate whitespace---as a multiplier.
 - Comments consist of anything one manages to enter, provided it doesn't start a new line.
The [.*] s indicate that a portion of the file could be omitted altogether:
 - Just because something is noted as optional doesn't mean it is in every case: the answer might
   actually depend on which other information is---or isn't---provided.
 - In particular, it is only legal to place a (CBFS) annotation on a leaf section; that is, choosing
   to add child sections excludes the possibility of putting a CBFS in their parent. Such
   annotations are only used to decide where CBFS empty file headers should be created, and do not
   result in the storage of any additional metadata in the resulting FMAP section.
Additionally, it's important to note these properties of the overall file and its values:
 - Other than within would-be strings and numbers, whitespace is ignored. It goes without saying
   that such power comes with responsibility, which is why this sentence is here.
 - Although the .*section names must be globally unique, one of them may---but is not required to---
   match the image name.
 - It is a syntax error to supply a number---besides 0---that begins with the character 0, as there
   is no intention of adding octals to the mix.
 - The image's memory address should be present on---and only on---layouts for memory-mapped chips.
 - Although it may be evident from above, all .*section offsets are relative only to the immediate
   parent. There is no way to include an absolute offset (i.e. from the beginning of flash), which
   means that it is "safe" to reorder the .*section s within a particular level of nesting, as long
   as the change doesn't cause their positions and sizes to necessitate overlap or zero sizes.
 - A .*section with omitted offset is assumed to start at as low a position as possible---with no
   consideration of alignment---and one with omitted size is assumed to fill the remaining space
   until the next sibling or before the end of its parent.
 - It's fine to omit any .*section 's offset, size, or both, provided its position and size are
   still unambiguous in the context of its *sibling* sections and its parent's *size*. In
   particular, knowledge of one .*section 's children or the .*section s' common parent's siblings
   will not be used for this purpose.
 - Although .*section s are not required to have children, the flash chip as a whole must have at
   least one.
 - Though the braces after .*section s may be omitted for those that have no children, if they are
   present, they must contain at least one child.

PL people and sympathizers may wish to examine the formal abstract syntax and context-free grammar,
which are located in fmd_scanner.l and fmd_scanner.y, respectively. Those interested in the
algorithm used to infer omitted values will feel at home in fmd.c, particularly near the definition
of validate_and_complete_info().