This patch makes libpayload enable the instruction cache as the very
first thing, which is similar to how we treat it in coreboot. It also
prevents the icache from being disabled again during mmu_disable() as
part of the two-stage page table setup in post_sysinfo_scan_mmu_setup().
It replaces the existing mmu_disable() implementation with the assembly
version from coreboot which handles certain edge cases better (see
CB:27238 for details).
The SCTLR flag definitions in libpayload seem to have still been
copy&pasted from arm32, so replace with the actual arm64 defintions from
coreboot.
Change-Id: Ifdbec34f0875ecc69fedcbea5c20e943379a3d2d
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://review.coreboot.org/c/coreboot/+/38908
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Hung-Te Lin <hungte@chromium.org>
This patch reworks the arm64 exception handling to be more similar to
how it works on arm32. This includes a bunch of features like actually
saving and restoring more exception state in the exception_state
structure and supporting the same sort of partial reentrancy that is
useful for GDB. Since there's no instruction to directly load into or
store out of SP on arm64, we can't do quite the same thing where we use
that to read an exception_state_ptr variable right after exception entry
when no other register is available. But we can do something very
similar by (ab-)using the "high" stack pointer (SP_EL2) as a pointer to
the exception_state struct and providing a function to change it.
Change-Id: Ia16a1124be1824392a309ae1f4cb031547d184c1
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://review.coreboot.org/29018
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
This patch adds the new, faster architectural register accessors to
libpayload that were already added to coreboot in CB:27881. It also
hardcodes the assumption that coreboot payloads run at EL2, which has
already been hardcoded in coreboot with CB:27880 (see rationale there).
This means we can drop all the read_current/write_current stuff which
added a lot of unnecessary helpers to check the current exception level.
This patch breaks payloads that used read_current/write_current
accessors, but it seems unlikely that many payloads deal with this stuff
anyway, and it should be a trivial fix (just replace them with the
respective _el2 versions).
Also add accessors for a couple of more registers that are required to
enable debug mode while I'm here.
Change-Id: Ic9dfa48411f3805747613f03611f8a134a51cc46
Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: https://review.coreboot.org/29017
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-by: Patrick Rudolph <patrick.rudolph@9elements.com>
Add: raw_read_cntfrq_el0() and raw_read_cntpct_el0()
Required to support Arch64 Timer
Change-Id: I86aa97039304b9e9336d0146febfe1811c9e075a
Signed-off-by: T Michael Turney <mturney@codeaurora.org>
Reviewed-on: https://review.coreboot.org/25649
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Julius Werner <jwerner@chromium.org>
The VA space needs to be extended to support 48bit, as on Cavium SoCs
the MMIO starts at 1 << 47.
The following changes were done to coreboot and libpayload:
* Use page table lvl 0
* Increase VA bits to 48
* Enable 256TB in MMU controller
* Add additional asserts
Tested on Cavium SoC and two ARM64 Chromebooks.
Change-Id: I89e6a4809b6b725c3945bad7fce82b0dfee7c262
Signed-off-by: Patrick Rudolph <patrick.rudolph@9elements.com>
Reviewed-on: https://review.coreboot.org/24970
Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
Reviewed-by: Julius Werner <jwerner@chromium.org>
Set XN bit of block upper attribute to device memory in mmu. CPU may
speculatively prefetch instructions from device memory, but the IO
subsystem of some implementation may not support this operation. Set
this attribute to device memory mmu entries can prevent CPU from
prefetching device memory.
BRANCH=none
BUG=none
TEST=build and booted to kernel on oak-rev3 with dcm enabled.
Change-Id: I52ac7d7c84220624aaf6a48d64b9110d7afeb293
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 7b01a4157cb046a5e75ea7625060a602e7a63c3c
Original-Change-Id: Id535e990a23b6c89123b5a4e64d7ed21eebed607
Original-Signed-off-by: Jimmy Huang <jimmy.huang@mediatek.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/302301
Original-Commit-Ready: Yidi Lin <yidi.lin@mediatek.com>
Original-Tested-by: Yidi Lin <yidi.lin@mediatek.com>
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Reviewed-by: Julius Werner <jwerner@chromium.org>
Reviewed-on: http://review.coreboot.org/11722
Tested-by: build bot (Jenkins)
Reviewed-by: Patrick Georgi <pgeorgi@google.com>
This applys the same change made by
https://chromium-review.googlesource.com/261692
to libpayload.
BUG=none
BRANCH=tot
TEST=built for veyron_jerry, rush_ryu, samus
Change-Id: I26dd66d79cd1559a7852b3c9d252420f2fed5fa0
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: d0d6f70aa805e18966e80618fbf9e9605274b030
Original-Change-Id: Ib0c199238f8fa58643d51782b17550dbd0d9ebd7
Original-Signed-off-by: Daisuke Nojiri <dnojiri@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/282541
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/10773
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
The current arm64 MMU interface is difficult to use in pre-RAM
environments. It is based on the memranges API which makes use of
malloc(), and early stages usually don't have a heap. It is also built
as a one-shot interface that requires all memory ranges to be laid out
beforehand, which is a problem when existing areas need to change (e.g.
after initializing DRAM).
The long-term goal of this patch is to completely switch to a
configure-as-you-go interface based on the mmu_config_range() function,
similar to what ARM32 does. As a first step this feature is added
side-by-side to the existing interface so that existing SoC
implementations continue to work and can be slowly ported over one by
one. Like the ARM32 version it does not garbage collect page tables that
become unused, so repeated mapping at different granularities will
exhaust the available table space (this is presumed to be a reasonable
limitation for a firmware environment and keeps the code much simpler).
Also do some cleanup, align comments between coreboot and libpayload for
easier diffing, and change all error cases to assert()s. Right now the
code just propagates error codes up the stack until it eventually
reaches a function that doesn't check them anymore. MMU configuration
errors (essentially just misaligned requests and running out of table
space) should always be compile-time programming errors, so failing hard
and fast seems like the best way to deal with them.
BRANCH=None
BUG=None
TEST=Compile-tested rush_ryu. Booted on Oak and hacked MMU init to use
mmu_config_range() insted of memranges. Confirmed that CRCs over all page
tables before and after the change are equal.
Change-Id: I93585b44a277c1d96d31ee9c3dd2522b5e10085b
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: f10fcba107aba1f3ea239471cb5a4f9239809539
Original-Change-Id: I6a2a11e3b94e6ae9e1553871f0cccd3b556b3e65
Original-Signed-off-by: Julius Werner <jwerner@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/271991
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/10304
Tested-by: build bot (Jenkins)
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
arch_program_segment_loaded ensures that the program segment loaded is
synced back from the cache to PoC. dcache_flush_all on arm64 does not
guarantee PoC in case of MP systems. Thus, it is important to track
and sync back all the required segments using
arch_program_segment_loaded.
BUG=chrome-os-partner:38231
BRANCH=None
TEST=Compiles successfully and boots to kernel prompt on smaug
Change-Id: Ic6fcc7e5e0cccbab317950f8abab0c494041d19a
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 284e3784854f764159b64286cea366c66b6bce2c
Original-Change-Id: I5c35b9aa2ae9b5c1f2fcdef40ffb1cde7f49cc1a
Original-Signed-off-by: Furquan Shaikh <furquan@google.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/263327
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Original-Trybot-Ready: Furquan Shaikh <furquan@chromium.org>
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/9904
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
In order to not duplicate the instruction cache invalidation
sequence provide a common routine to perform the necessary
actions. Also, use it in the appropriate places.
BUG=chrome-os-partner:38231
BRANCH=None
TEST=Compiles successfully for smaug and boots kernel
Change-Id: I1d311dbc70bf225f35d60bb10d8d001065322b3a
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 8ab015156713eb7531378edbd1d779522681d529
Original-Change-Id: I8da7002c56139f8f82503484bfd457a7ec20d083
Original-Signed-off-by: Furquan Shaikh <furquan@google.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/263326
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Original-Trybot-Ready: Furquan Shaikh <furquan@chromium.org>
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/9903
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Allow read/write to registers at a given el. Also, make read/write
registers at current el call this newly added function.
BRANCH=none
BUG=none
TEST=build and boot on mt8173-evb
Change-Id: Id69f0fdc07193c5c7e997712f0cd99de6f41510b
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: c091917babc39d9ab997f51f81b486c9aa900c24
Original-Change-Id: I0944946642066b88331e497a92388e74e86902d0
Original-Signed-off-by: HC Yen <hc.yen@mediatek.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/240322
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/8798
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Allocate noncacheable memory for frame buffer and save base
address to sys_libinfo.
BRANCH=none
BUG=chrome-os-partner:31936
TEST=build and test on ryu
Change-Id: I19a8079616376dc7c1a8ecdbd7499c2553b8c6c3
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: cebb5650167264902548339bb1a2b428f3b7f4ed
Original-Change-Id: I7bfbfefb92001632ce3d572a50e46188795c4ab8
Original-Signed-off-by: Jimmy Zhang <jimmzhang@nvidia.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/226404
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Reviewed-on: http://review.coreboot.org/8796
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
1. keep functions and objects used entirely within mmu.c as static.
2. DMA region finding needs to terminate. Therefore, the next address
to be attempted needs to be less then the current end address.
3. Ensure mmu_ranges passed to mmu_init_ranges_from_sysinfo() has
0 entries marked as used.
BUG=chrome-os-partner:31634
BRANCH=None
TEST=Booted ryu with RAM hole above cbmem tables below 4GiB.
Change-Id: I71a9cb89466978aa63fca5d8bee97b8af75ea206
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 66518fd86e676bbddf52e9d9afdd76d72c8e2222
Original-Change-Id: I5cb4e5009359cb04c4e1b5fe60845f80fbdff02c
Original-Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/221725
Original-Reviewed-by: Furquan Shaikh <furquan@chromium.org>
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/8793
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
In order to ease the process of reading and writing any register at current EL,
provide read_current and write_current assembly macros. These are included in
arch/lib_helpers.h under the __ASSEMBLY__ macro condition. This is done to allow
the same header file to be included by .c and .S files.
BUG=chrome-os-partner:31634
BRANCH=None
TEST=Compiles successfully for ryu
Change-Id: I79241a944b68ebb24865e745a9835f54ab6d1a8f
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 2b55fbde466126c4de7f5f7bb2d1427196be842f
Original-Change-Id: I678ab89c4aa1b08898166e135b5ab2d6453bb5e8
Original-Signed-off-by: Furquan Shaikh <furquan@google.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/214576
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/8801
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Adds support for initializing mmu, setting up dma areas and enabling mmu based
on the memranges passed on in the coreboot tables.
CQ-DEPEND=CL:216826
BUG=chrome-os-partner:31634
BRANCH=None
TEST=Compiles successfully
Change-Id: Id41a4255f1cd45a9455840f1eaa53503bd6fef3f
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: f2c6676bf51fcd85b61e9e08a261634a78137c4c
Original-Change-Id: I217bc5a5aff6a1fc0809c769822d820316d5c434
Original-Signed-off-by: Furquan Shaikh <furquan@google.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/216823
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/8791
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Add library helpers to access standard arm64 registers. This library also
provides functions to directly read/write register based on current el. So, rest
of the code doesnt need to keep checking the el and call appropriate function
based on that.
BUG=chrome-os-partner:31634
BRANCH=None
TEST=Libpayload and depthcharge compile successfully for ryu
Change-Id: Ibc0ca49f158362d4b7ab2045bf0fbd58ada79360
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 2ca6da580cb51b4c23abdaf04fee2785e5780510
Original-Change-Id: I9b63e04aa26a98bbeb34fdef634776d49454ca8d
Original-Signed-off-by: Furquan Shaikh <furquan@google.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/214575
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/8784
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Add support for memory barriers in arch {arm,arm64,x86}. This is required to
force strict CPU ordering. Definitions are based on FREEBSD atomic.h
definitions.
BUG=chrome-os-partner:31533
BRANCH=None
TEST=Memory barriers tested with ehci driver on arm64
Change-Id: I50060b0f33a6bd6cb95e829df079df379b2ff2a5
Signed-off-by: Patrick Georgi <pgeorgi@chromium.org>
Original-Commit-Id: 937d66cdab92a8521ede8307f5af8f5c20d3e552
Original-Change-Id: Ie51e3452f7a254b24111000da5dbe8714ac22223
Original-Signed-off-by: Furquan Shaikh <furquan@google.com>
Original-Reviewed-on: https://chromium-review.googlesource.com/213916
Original-Tested-by: Furquan Shaikh <furquan@chromium.org>
Original-Reviewed-by: Julius Werner <jwerner@chromium.org>
Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
Reviewed-on: http://review.coreboot.org/8731
Tested-by: build bot (Jenkins)
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
setbits_le32() is not really arch-specific... the arch-specific part of
accessing memory is wrapped by readl() and writel(), and the endianness
can be accounted for with the right macros. Generalize the definitions,
add a be32 version and move them to endian.h so that all platforms can
use them. Also include endian.h from libpayload.h so we won't update any
payload's old use of the macros (endianness is something useful enough
to always have avalable anyway, and shouldn't clash with other things).
This also fixes a bug where these macros would only be available if
libpayload-config.h had been independently included before.
Also fix a bug with readl() macros on all archs where they refused to
work on const pointers (which they should).
CQ-DEPEND=CL:208712
BUG=None
TEST=Stuff still compiles. Built and booted on Storm.
Original-Change-Id: I01a7fbadbb5d740675657d95c1e969027562ba8c
Original-Signed-off-by: Julius Werner <jwerner@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/208713
Original-Reviewed-by: Vadim Bendebury <vbendeb@chromium.org>
Original-Reviewed-by: David Hendricks <dhendrix@chromium.org>
(cherry picked from commit 951f8a6d77bc21bd793bf4f228a0965ade586f00)
Signed-off-by: Marc Jones <marc.jones@se-eng.com>
Change-Id: I51c25f01b200b91abbe32c879905349bb05dc9c8
Reviewed-on: http://review.coreboot.org/8129
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
Tested-by: build bot (Jenkins)
This patch makes some slight changes to the exception hook interface.
The old code provides a different handler hook for every exception
type... however, in practice all those hook functions often need to look
very similar, so this creates more boilerplate than it removes. The new
interface just allows for a single hook with the exception type passed
as an argument, and the consumer can signal whether the exception was
handled through the return value. (Right now this still only supports
one consumer, but it could easily be extended to walk through a list of
hooks if the need arises.)
Also move the excepton state from an argument to a global. This avoids a
lot of boilerplate since some consumers need to change the state from
many places, so they would have to pass the same pointer around many
times. It also removes the false suggestion that the exception state was
not global and you could have multiple copies of it (which the exception
core doesn't support for any architecture).
On the ARM side, the exception state is separated from the exception
stack for easier access. (This requires some assembly changes, and I
threw in a few comments and corrected the immediate sigils from '$' to
the official '#' while I'm there.) Since the exception state is now both
stored and loaded through an indirection pointer, this allows for some
very limited reentrance (you could point it to a different struct while
handling an exception, and while you still won't be able to return to
the outer-level exception from there, you could at least swap out the
pointer and return back to System Mode in one go).
BUG=chrome-os-partner:18390
TEST=Made sure normal exceptions still get dumped correctly on both
archs.
Original-Change-Id: I5d9a934fab7c14ccb2c9d7ee4b3465c825521fa2
Original-Signed-off-by: Julius Werner <jwerner@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/202562
Original-Reviewed-by: Stefan Reinauer <reinauer@chromium.org>
(cherry picked from commit 97542110f0b385b9b8d89675866e65db8ca32aeb)
Signed-off-by: Marc Jones <marc.jones@se-eng.com>
*** Squashed to prevent build failures. ***
libpayload: align arm64 with new exception handling model
The exception handling was previously updated, however the
arm64 changes raced with hat one. Make the arm64 align with
the new model. Without these changes compilation will fail.
BUG=None
BRANCH=None
TEST=Can build libpayload for rush.
Original-Change-Id: I320b39a57b985d1f87446ea7757955664f8dba8f
Original-Signed-off-by: Aaron Durbin <adurbin@chromium.org>
Original-Reviewed-on: https://chromium-review.googlesource.com/204402
Original-Reviewed-by: Furquan Shaikh <furquan@chromium.org>
Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org>
(cherry picked from commit 0080df41b311ef20f9214b386fa4e38ee54aa1a1)
Signed-off-by: Marc Jones <marc.jones@se-eng.com>
Change-Id: I9a0bb3848cf5286f9f4bb08172a9f4a15278348e
Reviewed-on: http://review.coreboot.org/8117
Tested-by: build bot (Jenkins)
Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
Julius Werner