_____ ____ _____ ______ ____ ____ ____ _______ / ____/ __ \| __ \| ____| _ \ / __ \ / __ \__ __| | | | | | | |__) | |__ | |_) | | | | | | | | | | | | | | | _ /| __| | _ <| | | | | | | | | | |___| |__| | | \ \| |____| |_) | |__| | |__| | | | \_____\____/|_| \_\______|____/ \____/ \____/ |_| __ __ _____ _____ ____ /\ | \/ | __ \ / ____| |___ \ / \ | \ / | | | | | (___ __) | / /\ \ | |\/| | | | | \___ \ |__ < / ____ \| | | | |__| | ____) | ___) | /_/ \_\_| |_|_____/ |_____/ |____/ S3 in coreboot (V 1.2) ---------------------------------------- Zheng Bao <zheng.bao@amd.com> <fishbaozi@gmail.com> Introduction ============ This document is about how the feature S3 is implemented on coreboot, specifically on AMD platform. This topic deals with ACPI spec, hardware, BIOS, OS. We try to help coreboot users to realize their own S3. S3 in a nutshell ================ The S3 sleeping state is a low wake latency sleeping state where all system context is lost except system memory. [1]. S3 is a ACPI definition. To enter S3, write 3 in SLP_TYPx and set the SLP_EN bit (See ACPI registers). But if you do that, board can not resume at where it sleeps, because you don't save the context. More often than not, we make the board go into S3 by the tools which OSes provide. For windows, click Start->sleep. For linux, some distribution provide a tools called pm-suspend, which can make the system goto S3. If pm-suspend is not available, we can run "echo mem > /sys/power/state", but this way may not save all the needed context. In S3 state, the power is off. So when the power button is pressed, BIOS runs as it does in cold boot. If BIOS didn't detect whether board boots or resumes, it would go the same way as boot. It is not what we expect. BIOS detects the SLP_TYPx. If it is 3, it means BIOS are waking up. BIOS is responsible for restore the machine state as it is before sleep. It needs restore the memory controller, not overwriting memory which is not marked as reserved. For the peripheral which loses its registers, BIOS needs to write the original value. When everything is done, BIOS needs to find out the wakeup vector provided by OSes and jump there. OSes also have work to do. We can go to linux kernel or some other open source projects to find out how they handle S3 resume. ACPI registers ============== ACPI specification defines a group of registers. OSes handle all these registers to read and write status to all the platform. On AMD platform, these registers are provided by southbridge. For example, Hudson uses PMIO 60:6F to define ACPI registers. OSes don't have any specific driver to know where these registers are. BIOS has the responsibility to allocated the IO resources and write all these address to FADT, a ACPI defined table. Memory Layout ============= Restoring memory is the most important job done by BIOS. When the power is off, the memory is maintained by standby power. BIOS need to make sure that when flow goes to OS, everything in memory should be the same as it was. The chip vendor will provide a way, or code, to wake up the memory from sleeping. In AGESA 2008 arch, it is called AmdInitResume. The BIOS itself needs some memory to run. Either, BIOS marks the erea as reserved in e820, or BIOS saves the content into reserved space. Here is the address Map for S3 Resume. Assumingly the total memory is 1GB. 00000000 --- 00100000 BIOS Reserved area. 00100000 --- 00200000 Free 00200000 --- 01000000 coreboot ramstage area. 01000000 --- 2e160000 Free 2e160000 --- 2e170000 ACPI table 2e170000 --- 2ef70000 OSRAM 2ef70000 --- 2efe0000 Stack in highmem 2efe0000 --- 2f000000 heap in highmem 2f000000 TOM AMD requirements in S3 ====================== Chip vendor like AMD will provide bunch of routines to restore the board.[2] * AmdS3Save: It is called in cold boot, save required register into non-volatile storage. Currently, we use SPI flash to store the data. * AmdInitResume: Restore the memory controller. * AmdS3LateRestore: Called after AmdInitResume, restore other register that memory. * (SouthBridge)InitS3EarlyRestore, (SouthBridge)InitS3LateRestore: Provided by Southbridge vendor code. Early is called before PCI enumeration, and Late is called after that. Lifecycle of booting, sleeping and waking coreboot and Ubuntu ============================================================= 1. Cold boot. For a system with S3 feature, the BIOS needs to save some data to non-volatile storage at cold boot stage. What data need to be save are provided by AmdS3Save. After the wrapper calls the AmdS3Save, it gets the VolatileStorage and NvStorage, which are where the data are located. It is the wrappers's responsibility to save the data.[3][4] Currently, the wrappers allocate a CBFS modules in BIOS image. To do that, the wrapper needs to have the ability to write flash chips. It is not as comprehensive as flashrom. But for the SST chip on Parmer, MX chip on Thather, coreboot works well.[5] 2. OS goes in S3. For Linux, besides the kernel needs to do some saving, most distributions run some scripts. For Ubuntu, scripts are located at /usr/lib/pm-utils/sleep.d. # ls /usr/lib/pm-utils/sleep.d 000kernel-change 49bluetooth 90clock 95led 00logging 55NetworkManager 94cpufreq 98video-quirk-db-handler 00powersave 60_wpa_supplicant 95anacron 99video 01PulseAudio 75modules 95hdparm-apm The script with lower prefix runs before the one with higher prefix. 99video is the last one. Those scripts have hooks called hibernate, suspend, thaw, resume. For each script, suspend is called when system sleeps and wakeup is called when system wakeups. 3. Firmware detects S3 wakeup As we mentioned, Firmware detects the SLP_TYPx to find out if the board wakes up. In romstage.c, AmdInitReset and AmdInitEarly are called as they are during cold boot. AmdInitResume and AmdS3LateRestore are called only during resume. For whole ramstage, coreboot goes through almost the same way as cold boot, other than not calling the AmdInitMid, AmdInitLate and AmdS3Save, and restoring all the MTRRs. At last step of coreboot stage, coreboot finds out the wakeup vector in FADT, written by OS, and jump. 4. OS resumes. When Linux resumes, all the sleeping scripts call their resume hooks. If we are more lucky, all the scripts can go through. More chances that the 99video hangs or fails to get the display back. Sometimes it can fixed if CONFIG_S3_VGA_ROM_RUN is unset in coreboot/Kconfig. That needs more troubleshooting. Reference ========= [1] ACPI40a, http://www.acpi.info/spec40a.htm [2] coreboot Vendorcode, {top}/src/vendorcode/amd/agesa/{family}/Proc/Common/ [3] coreboot AGESA wrapper, {top}/src/mainboard/amd/parmer/agesawrapper.c [4] coreboot AGESA wrapper, {top}/src/cpu/amd/agesa/s3_resume.c [5] coreboot Southbridge, {top}/src/southbridge/amd/agesa/hudson/spi.c